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1 . The designated Office is hereby notified of its election made: 

I X I in the demand filed with the International Preliminary Examining Authority on: 

14 May 1999 (14.05.99) 



I I in a notice effecting later election filed with the International Bureau on: 



2. The election 



□ 



was not 



made before the expiration of 19 months from the priority date or, where Rule 32 applies, within the time limit under 
Rule 32.2(b). 



The International Bureau of WlPO 
34, chemin des Colombettes 
1211 Geneva 20, Switzerland 



Facsimile No.: (41-22) 740.14.35 



Authorized officer 

F. Baechler 
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THIS PAGE BUNK (usPTO) 



VERTRACj 



ER DIE INTERNATIONALE Z\Ji 
F DEM GEBIET DES PATENTWJ 



PCT 

INTERNATIONALER RECHERCHENBERICHT 

(Artikel 18 sowie Regein 43 und 44 PCT) 



ENARBEIT 
NS 



Aktenzeichen des Anmelders Oder Anwalts 
Bw 19752056 


WEITERES siehe MitteHung iiber die Ubermittlung des internationalen 

Recherchenberichts (Formblatt PCT/ISA/220) sowie, soweit 
VORGEHEN zutreffend, nachstehender Punkt 5 


Internationales Aktenzeichen 

PCT/EP 98/07571 


Internationales Anmeldedatum 

(Tag/Monat/Jahr) 

24/1 1/1998 


(Fruhestes) Prioritatsdatum (Tag/Monat/Jahr) 

25/11/1997 


Anmelder 

BAYERISCHE MOTOREN WERKE AKTIENGESELLSCHAFT et al . 



Dieser internationale Recherchenbericht wurde von der Internationalen Recherchenbehorde erstellt und wird dem Anmelder gemaB 
Artikel 18 ubermittelt. Eine Kople wird dem Internationalen Buro Cibermittelt. 

Dieser internationale Recherchenbericht umfafBt insgesamt _4 Blatter. 

Pn DarCiber hinaus liegt ihm jeweils eine Kopie der in diesem Bericht genannten Unterlagen zum Stand der Technik bei. 



1 . Grundlage des Berichts 

a. Hinsichtlich der Sprache ist die internationale Recherche auf der Grundlage der internationalen Anmeldung in der Sprache 
durchgefuhrt worden. In der sie eingereicht wurde. sofern unter diesem Punkt nichts anderes angegeben ist. 

I I Die Internationale Recherche ist auf der Grundlage einer bei der Behdrde eingereichten Obersetzung der internationalen 
Anmeldung (Reget 23.1 b)) durchgefuhrt worden. 

b. Hinsichtlich der in der internationalen Anmeldung offenbarten Nucleotid- und/oder Aminosauresequenz ist die internationale 
Recherche auf der Grundlage des Sequenzprotokolls durchgefuhrt worden, das 

I I in der Internationalen Anmeldung in Schriflicher Form enthalten ist. 

I I zusammen mit der internationalen Anmeldung in computerlesbarer Form eingereicht worden ist. 

I I bei der Behorde nachtraglich in schriftlicher Form eingereicht worden ist. 

I I bei der Behorde nachtraglich in computerlesbarer Form eingereicht worden ist. 

I I Die Erkiarung, da3 das nachtraglich eingereichte schriftliche Sequenzprotokoll nicht uber den Offenbarungsgehalt der 
Internationalen Anmeldung im Anmeldezeitpunkt hinausgeht, wurde vorgetegt. 

I I Die Erklarung, da3 die in computerlesbarer Form erfa3ten Informationen dem schhftlichen Sequenzprotokoll entsprechen, 
wurde vorgelegt. 

2. Bestimmte Anspruche haben sich als nicht recherchierbar erwiesen (siehe Feld I). 

3. Mangelnde Einheitlichkeit der Erfindung (siehe Feld II). 



Hinsichtlich der Bezetchnung der Erfindung 

[X] wird der vom Anmelder eingereichte Wortlaut genehmigt. 
I I wurde der Wortlaut von der Behorde wie folgt festgesetzt: 



Hinsichtlich der Zusammenfassung 

I I wird der vom Anmelder eingereichte Wortlaut genehmigt. 

wurde der Wortlaut nach Regel 38.2b) in der in Feld III angegebenen Fassung von der Behorde festgesetzt. Der 
I X I Anmelder kann der Behdrde Innerhalb eines Monats nach dem Datum der Absendung dieses internationalen 

Recherchenberichts eine Steltungnahme vorlegen. 

Folgende Abbildung der Zeichnungen ist mit der Zusammenfassung zu ver6ffentiichen: Abb. Nr. \ 



I I wie vom Anmelder vorgeschlagen Q kelne der Abb. 

PC] well der Anmelder selbst keine Abbildung vorgeschlagen hat. 
I I well diese Abbildung die Erfindung besser kennzeichnet. 



Formblatt PCT/ISA/210 (Blatt 1) (Juli 1998) 



ioWrler RECHERCHENBERICHT 



T|ternationales Aktenzeichen 
PCT/EP 98/07571 



Feld III WORTLAUT DER ZUSAMMENFASSUNG (Fortsetzung von Punkt 5 auf Blatt 1) 



Die Zusammenf assung wird wie folgt geandert: 

Zeile 8: nach "Randberei ch" ist "(2)" einzufugen; 
Zeile 12: nach "Mittelbereich" ist "(3)" einzufugen. 



Formblatt PCT/ISA/210 (Fortsetzung von Blatt 1 (2))(Juli 1998) 



INTERNATIONALEB-RECHERCHENBERICHT 



lationales Aktenzeichen 

/EP 98/07571 



A. KLASSIFIZIERUNG OES ANMELOUNGSGEGENSTANDES 

IPK 6 G06F3/033 



Nach der Internationalen Patentklassifikation (IPK) oder nach der nationalen Klassifikation und der IPK 



B. RECHERCHIERTE GEBIETE 



Recherchierter Mindestprufstoff (Klassifikationssystem und Klassifikationssymbole ) 

IPK 6 G06F 



Recherchierte aber nicht zum Mindestprufstoff gehbrende Veroffentlichungen, soweit diese unter die recherchierter Gebiete fallen 



Wahrend der internationalen Recherche konsultierte elektronische Datenbank (Name der Datenbank und evtl. verwendete Suchbegriffe) 



C. ALS WESENTLICH ANGESEHENE UNTERLAGEN 



Kategorie° Bezeichnung der Veroffentlichung, soweit erforderlich unter Angabe der in Betracht kommenden Teile 



Betr. Anspruch Nr. 



EP 0 796 766 A (PHILIPS PATENTVERWALTUNG 

;PHILIPS ELECTRONICS NV (NL)) 

24. September 1997 

in der Anmeldung erwahnt 

siehe das ganze Dokument 



WO 96 30822 A (FOREST DONALD K) 
3. Oktober 1996 

siehe Seite 42, Zeile 18 - Zeile 
siehe Seite 86, Zeile 15 - Zeile 
siehe Abbildung 19 
siehe Abbildungen 52,53,64,65 



34 
25 



US 4 794 388 A (MATTHEWS HENRY G) 
27. Dezember 1988 

siehe Spalte 11, Zeile 45 - Spalte 12, 
Zeile 37; Anspruche 1-3; Abbildungen 1,4 

-/-- 



2-6 

1 

7 

1-6 



Weitere Veroffentlichungen sind der Fortsetzung von Feld C zu 
entnehmen 



0 



Siehe Anhang Patentfamilie 



° Besondere Kategorien von angegebenen Veroffentlichungen 'T' 

"A" Veroffentlichung, die den allgemelnen Stand der Technik definiert, 
aber nicht als besonders bedeutsann anzusehen ist 

"E" alteres Dokument, das jedoch erst am Oder nach dem internationalen 
Anmeldedatum veroffentlicht worden ist 

"L" Veroffentlichung, die geeignet ist, einen Prioritatsanspruch zweifelhaft er- 
scheinen zu lassen, oder durch die das Veroffentlichungsdatum einer 
anderen im Recherchenbericht genannten Veroffentlichung belegt werden ..y, 
soil Oder die aus einem anderen besonderen Grund angegeben ist (wie 
ausgefuhrt) 

"O" Veroffentlichung, die sich auf eine mundliche Offenbarung, 

eine Benutzung, eine Ausstellung oder andere MaRnahmen bezieht 
"P" Veroffentlichung, die vor dem internationalen Anmeldedatum, aber nach 
dem beanspruchten Prioritatsdatum veroffentlicht worden ist 



Spatere Veroffentlichung, die nach dem internationalen Anmeldedatum 
Oder dem Prioritatsdatum veroffentlicht worden ist und mtt der 
Anmeldung nicht kollidiert, sondern nur zum Verstandnis des der 
Erfindung zugrundeliegenden Prinzips oder der ihr zugrundeliegenden 
Theorie angegeben ist 

Veroffentlichung von besonderer Bedeutung; die beanspruchte Erfindung 
kann allein aufgrund dieser Veroffentlichung nicht als neu oder auf 
erfinderischer Tatigkeit beruhend betrachtet werden 

Veroffentlichung von besonderer Bedeutung; die beanspruchte Erfindung 
kann nicht als auf erfinderischer Tatigkeit beruhend betrachtet 
werden, wenn die Veroffentlichung mit einer oder mehreren anderen 
Veroffentlichungen dieser Kategorie in Verbindung gebracht wird und 
diese Verbindung fur einen Fachmann naheliegend ist 

Veroffentlichung, die Mitglied derselben Patentfamilie ist 



Datum des Abschlusses der internationalen Recherche 

30. Marz 1999 


Absendedatum des internationalen Recherchenberichts 

08/04/1999 


Name und Postanschrift der Internationalen Recherchenbehorde 
Europaisches Patentamt, P. B. 5818 Patentlaan 2 
NL - 2280 HV Rijswijk 
Tel. (+31-70) 340-2040, Tx. 31 651 epo nl. 
Fax: (+31-70) 340-3016 


Bevollmachtigter Bedlensteter 

Duran(J, J 
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INTERNATIONALER RECHERCHENBERICHT 



rnationales Aktenzeichen 

iT/EP 98/07571 



C.<Fortsetzung) ALS WESENTLICH ANGESEHENE UNTERLAGEN 



Kategorie' 



Bezeichnung der Veroffentlichung, soweit erforderlich unter Angabe der in Betracht kommenden Teile 



Betr Anspruch Nr. 



EP 0 498 082 A (KONINKL PHILIPS 
ELECTRONICS NV) 12. August 1992 
siehe Spalte 6, Zeile 22 - Zeile 35; 
Abbildungen 3,7 

US 5 297 253 A (MEISEL LESLIE M) 
22. Marz 1994 

siehe Spalte 7, Zeile 9 - Zeile 37; 
Abbi Idungen 

PATENT ABSTRACTS OF JAPAN 
vol. 097, no. 009, 30. September 1997 
& OP 09 114902 A (HITACHI LTD;HITACHI 
SEIBU SOFTWARE LTD), 2. Mai 1997 
siehe Zusammenfassung 

"ALGORITHM FOR DECREASING THE ERROR RATE 
OF DATA ENTERED ON A TOUCH-SENSITIVE 
TERMINAL" 

IBM TECHNICAL DISCLOSURE BULLETIN, 
Bd. 33, Nr. lOA, 1. Marz 1991, Seiten 
223-227, XP000110024 
siehe Seite 224 

EP -0 623 870 A (IBM) 9. November 1994 
siehe das ganze Dokument 

EP 0 366 132 A (BAYERISCHE MOTOREN WERKE 

AG) 2. Mai 1990 

in der Anmeldung erwahnt 

siehe das ganze Dokument 



1,7 



8 

1,7 
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INTERNATIONAL SEARCH REPORT 

(tion on patent family members 



national Application No 

WeP 98/07571 



Patent document 
cited in search report 


Publication 
date 


Patent family 
member(s) 


Publicabon 
date 


EP 0796766 A 


24-09-1997 


DE 


19610344 


A 


25-09-1997 






JP 


10020950 


A 


23-01-1998 






US 


5883346 


A 


16-03-1999 



WO 9630822 A 03-10-1996 AU 2191095 A 16-10-1996 

DE 19581933 T 16-04-1998 
GB 2314186 A 17-12-1997 



US 4794388 A 27-12-1988 NONE 



EP 0498082 


A 


12-08-1992 


DE 


69129364 


D 


10-06-1998 








DE 


69129364 


T 


22-10-1998 








JP 


4317119 


A 


09-11-1992 








US 


5828360 


A 


27-10-1998 



US 5297253 A 22-03-1994 NONE 



EP 0623870 A 09-11-1994 CA 2095452 A 05-11-1994 

JP 6332648 A 02-12-1994 

US 5559945 A 24-09-1996 

US 5689668 A 18-11-1997 



EP 0366132 


A 


02-05-1990 


DE 


3836555 


A 


10-05-1990 








DE 


4001062 


A 


18-07-1991 








DE 


58908735 


D 


19-01-1995 








ES 


2064418 


T 


01-02-1995 








JP 


2187814 


A 


24-07-1990 








JP 


2840332 


B 


24-12-1998 








US 


5270689 


A 


14-12-1993 
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PATENT COOPERATION TREATY 

PCT 

INTERNATIONAL PRELIMINARY EXAMINATION REPORT <^-Q 
(PCT Article 36 and Rule 70) 



Applicant's or agent's file reference 
Bw 19752056 


F-nu FirRTHirR ArriON Notification of Transmittal of International 
hUK hUKlMiLK ACiiUfN Preliminary Examination Report (Form PCT/IPEA/416) 


International application No. 

PCT/EP98/07571 


International filing date {day/month/year) 
24 November 1998 (24.1 1.98) 


Priority date (day/month/year) 

25 November 1 997 (25 . 1 1 .97) 


International Patent Classification (IPC) or national classification and IPC 
G06F 3/033 


Applicant 

BAYERISCHE MOTOREN WERKE AKTIENGESELLSCHAFT 



This international preliminary examination report has been prepared by this International Preliminary Examining 
Authority and is transmitted to the applicant according to Article 36. 



This REPORT consists of a total of 



. sheets, including this cover sheet. 



I I This report is also accompanied by ANNEXES, i.e., sheets of the description, claims,and/or drawings which have 
' — ' been amended and are the basis for this report and/or sheets containing rectifications made before this Authority 
(see Rule 70. 1 6 and Section 607 of the Administrative Instructions under the PCT). 

These annexes consist of a total of sheets. 



This report contains indications relating to the following items: 
I [)x(] Basis of the report 
Priority 

Non-establishment of opinion with regard to novelty, inventive step and industrial applicability 
Lack of unity of invention 

Reasoned statement under Article 35(2) with regard to novelty, inventive step or industrial applicability: 
citations and explanations supporting such statement 

Certain documents cited 

Certain defects in the international application 

Certain observations on the international application 



II 


□ 


III 


□ 


IV 


□ 


V 




VI 


□ 


vu 




VIII 


□ 



Date of submission of the demand 

14 May 1999(14.05.99) 


Date of completion of this report 

1 7 January 2000 (17.01 .2000) 


Name and mailing address of the IPEA/EP 
Facsimile No. 


Authorized officer 
Telephone No. 



Form PCT/IPEA/409 (cover sheet) (January 1994) 



INTERNATIONAL PRELIMINARY EXAMINATION REPORT 



International application No. 

PCT/EP98/07571 



I. Basis of the report 



1 . This report has been drawn on the basis of {Replacement sheets which have been Jumished to the receiving Office in response to an invitation 
under Article 14 are referred to in this report as "originally filed" and are not annexed to the report since they do not contain amendments ). 

I I the international application as originally filed. 

the description, pages j.:^ ^ as originally filed, 

pages , filed with the demand, 

pages , filed with the letter of 

pages , filed with the letter of 



the claims, 



Nos, 
Nos. 
Nos. 
Nos. 
Nos. 



1-8 



as originally filed, 

as amended under Article 19, 

filed with the demand, 

filed with the letter of 

filed with the letter of 



^ the drawings, 



sheets/fig 
sheets/fig 
sheets/fig 
sheets/fig 



1/3-3/3 



, as originally filed, 
, filed with the demand, 
, filed with the letter of 
, filed with the letter of 



2. The amendments have resulted in the cancellation of: 

I I the description, pages 

I I the claims. Nos. 



I I the drawings, sheets/fig 



I I This report has been established as if (some oO the amendments had not been made, since they have been considered 
' — ' to go beyond the disclosure as filed, as indicated in the Supplemental Box (Rule 70.2(c)). 



4. Additional observations, if necessary: 



Form PCT/IPE A/409 (Box I) (January 1994) 



INTERNATIONAL PRELIMINARY EXAMINATION REPORT 



'international application No. 
PCT/EP 98/07571 



V. Reasoned statement under Article 35(2) with regard to novelty, inventive step or industrial applicability; 
citations and explanations supporting such statement 

1 . Statement 

Novelty (N) Claims 1*8 

Claims 

1 — R 

Inventive step (IS) Claims ^ 

Claims 

Industrial applicabilit>' (lA) Claims ^"^ YES 

Claims NO 

2. Citations and explanations 

Reference is made to the following documents: 

Dl: EP-A-0 796 766 (PHILIPS PATENTVERWALTUNG; PHILIPS 
ELECTRONICS NV (NL) ) , 24 September 1997, mentioned 
. - ^ - - - -in- the application 

D2: WO 96 30822 A (FOREST DONALD K) , 3 October 1996 

1 The invention concerns a device for controlling a 
display screen with an actuating member. 

2 Document Dl discloses a device such as this. The 
actuating member serves to increase the number of 
levels within the menu structure. To this effect, it 
is stated that the levels should be distributed over 
the extent of the display screen. 

A device such as this adds an additional 
complication for the user, since there is no clear 
co-ordination between the additional movement and 
the axial or rotational movement of the actuating 
member. The transition between the levels and within 
the menu structure is not clear and this leads 
ultimately to the driver being considerably 
distracted from what is happening on the road. 

Form PCT/IPEA/409 (Box V) (Januarj' 1994) 



YES 
NO 
YES 
NO 



INTERNATIONAL PRELIMINARY EXAMINATION REPORT 



Intemationai application No. 
PCT/EP 98/07571 



The essential difference of the invention with 
respect to the prior art is that there is a clear 
co-ordination between the display screen and the 
different possibilities for moving the actuating 
member . 



The additional movement of the actuating member 
causes a superior point of the menu structure to be 
selected, and the rotational or lengthwise movement 
of the actuating member causes a subordinate point 
of the menu structure to be selected. The possible 
menus are arranged in the edge area, whilst the 
corresponding sub-menus, functions or function 
values are located in the middle area. 



Form PCT/IPEA/409 (Box V) (January 1994) 



INTERNATIONAL PRELIMINARY EXAMINATION REPORT 



kemational application No. 
PCT/EP 98/07571 



VII. Certain defects in the international application 



The following defects in the form or contents of the international application have been noted: 



The features of the preamble of the claims have not been 
provided with reference signs placed between parentheses 
(PCT Rule 6.2 (b) ) . 



Form PCT/lPEA/409 (Box VII) (January 1994) 



VERTRAG UBER 



V 



^ INTERNATIONALE ZUSAr^^NARBEIT AUF DEM 
GEBIET DES PATENTWESENS 



PCT 

INTERNATIONALER VORLAUFIGER PRUF 

(Artikel 36 und Regel 70 PCT) 




Aktenzeichen des Anmelders oder Anwalts 
AJ-33/Bw/dr/PA 1 975 



Internationales Aktenzeichen 
PCT/EP98/07571 



WEITERES VORGEHEN 



siehe Mitteilung uber die Ubersendung des internationalen 
vorlaufigen Prufungsbericht (Fomnblatt PCT/IPEA/416) 



Internationales Pinme\deda\urr\(Tag/Monat/Jahr) 
24/11/1998 



Prioritatsdatum (Tag/Monat/Tag) 
25/11/1997 



Internationale Patentklassification (IPK) oder nationale Klassifikation und IPK 
G06F3/033 



Anmelder 

BAYERISCHE MOTOREN WERKE AKTIENGESELLSCHAFT et al. 



1 . Dieser internationale vorlaulige Prufungsbericht wurde von der m\X der internationals vorlaufigen Prufung beauftragte 
Behorde erstellt und wird dem Anmelder gema3 Artikel 36 ubermrttelt. 

2. Dieser BERICHT umfaBt insgesamt 5 Blatter einschlieBlich dieses Deckblatts. 

□ AuBerdem liegen dem Bericht ANLAGEN bei; dabei handelt es sich urn Blatter mit Beschreibungen. Anspruchen 
und/oder Zeichnungen, die geandert wurden und diesem Bericht zugrunde liegen. und/oder Blatter mit vor dieser 
Behorde vorgenommenen Berichtigungen (siehe Regel 70.16 und Abschnitt 607 der Venwaltungsrichtlinien zum PCT) 

Diese Aniagen umfassen Insgesamt Blatter. 



3. Dieser Bericht enthalt Angaben zu folgenden Punklen: 

I H Grundlage des Berichts 
li □ Prioritm 

III □ Keine Erstellung eines Gutachtens uber Neuhelt. erlinderische Tatigkeit und gewerbliche Anwendbarkeit 

IV □ Mangelnde Einheitlichkeit der Erfindung 

V S Begrundete Feststellung nach Artikel 35(2) hinsichtlich der Neuheit. der erfinderische Tatigkeit und der 

gewerbliche Anwendbarkeit; Unterlagen und ErklSrungen zur StOtzung dieser Feststellung 

VI □ Bestimmte angefQhrte Unterlagen 

VII H Bestimmte Mangel der internationalen Anmeldung 

VIII □ Bestimmte Bemerkungen zur internationalen Anmeldung 



Datum der Einreichung des Antrags 
14/05/1999 


Datum der Fertigstellung dieses Berichts 

1 7. 01. 00 


Name und Postanschrift der mit der intemationalen vorlaufigen 
Prufung beauftragten Beh6rde: 

EuropSisches Patentamt 

^ D-80298 Munchen 

Tel. +49 89 2399 - 0 Tx: 523656 epmu d 
Fax: +49 89 2399 - 4465 


Bevollmachtigter Bediensteter z;^^*"*^^*^^^ 

1 M 1 

Alonso V Goicolea, L \* sO^' JJ 

Tel. Nr. +49 89 2399 7475 



Formblatt PCT/IPEA/409 (Deckblatt) (Januar 1994) 



INTERNATIONALER VORLAUFIGER Dr-T/nPQR/nT^yi 
PROFUNGSBERICHT Internationales Aktenzcchen PCT/EP98/07571 



I Grundlage des Berichts 

, , Oieser B.Hch. wurd. au. d,r Gru™..^ ^''^S^^f^'^^.T. ^^^2 ^^.Z^T^TTS 

Artikel 14 hin vorgelegt warden, gelten im Rahmen dieses Benefits als ursprungncn umy^f 
nicht beigefugt, weil sie keine Anderungen enthalten.): 

Beschreibung, Seiten: 

ursprungliche Fassung 

Patentanspriiche, Nr.: 

-|.8 ursprungliche Fassung 

Zeichnungen, Blatter: 

1/3-3/3 ursprungliche Fassung 



2. Aufgrund der Anderungen sind folgende Unterlagen fortgefallen: 



□ Beschreibung, Seiten: 

□ Anspruche, Nr.: 

□ Zeichnungen, Blatt: 



eingereichten Fassung hinausgehen (Regei 70.2(c)): 



4. Etwaige zusatzliche Bemerkungen: 



1. Feststellung 
Neuheit (N) 

Erfinderische Tatigkeit (ET) 
Gewerbliche Anwendbarkeit (GA) 



Ja: Anspruche 1 -8 
Nein: Anspruche 

Ja: Anspruche 1-8 
Nein: Anspruche 

Ja: Anspruche 1 -8 
Nein: Anspruche 



Formblatt PCT/IPEA/409 (Felder l-VIII. Blatt 1) (Januar 1994) 



INTERNATIONALER VORLAUFIGER 
PRUFUNGSBERICHT 



Internationales Aktenzeichen PCT/EP98/07571 



2. Unterlagen und Erklarungen 
siehe Beiblatt 



VII. Bestimmte Mangel der Internationalen Anmeidung 

Es wurde festgestellt. daB die internationale Anmeidung nach Form oder Inhalt folgende Mangel aufweist: 
siehe Beiblatt 



Formblatt PCT/lPEA/409 (Felder l-VIII. Blatt2) (Januar 1994) 



INTERNATIONALER VORLAUFIGER 
PRUFUNGSBERICHT - BEIBLATT 



Internationales Aktenzeichen PCT/EP98/07571 



Beg^rndete Feststellung nach Artikel 35(2) hinsichtlich der Neuheit, der 
ertfnderischen Tatigkeit und der gewerblichen Anwendbarkeit; Unter.agen und 
Erklarungen zur Stiitzung dieser Feststellung. 

Es wird auf die folgenden Dokumente verwiesen: 

D1- EP-A-0 796 766 (PHILIPS PATENTVERWALTUNG; PHILIPS 

ELECTRONICS NV (NL)) 24. September 1997 in der Anmeldung erwahnt 
D2- WO 96 30822 A (FOREST DONALD K) 3. Oktober 1996 



1 



Die Erfindung betrifft eine Vorrichtung zur Steuerung einer Bildschirmanzeige mit 
einem Betatigungsglied. 

Dokument D1 offenbart eine derartige Vorrichtung. Dabei dient das 
Betatigungslied dazu, die Anzahl der Ebenen innerhalb der Menustruktur zu 
vergroBern. Hierzu ist angegeben. die Ebenen uber den Umfang der 
Bildschirmanzeige verteilt anzuordnen. 

Eine derartige Vorrichtung bedeutet fur den Benutzer eine zusatzliche 
Schwierigkeit, da er keine eindeutige Zuordnung zwischen der Zusatzbewegung 
und der Axial- bzw. Rotationsbewegung des Betatigungslieds vorf.ndet. Auch 
gestaltet sich der Ubergang zwischen den Ebenen und innerhalb der 
Menustruktur fur ihn nicht eindeutig und fuhrt letztendlich zu einer erhebl.chen 
Ablenkung des Benutzers vom Verkehrsgeschehen. 

Gegenuber dem Stand der Technik hat die Erfindung den wesentlichen 
Unterschied. daB bei ihr eine eindeutige Zuordnung zwischen der 
Bildschirmanzeige und den verschiedenen Bewegungsmoglichkeiten des 
Betatigungsglieds besteht. 

Durch die Zusatzbewegung des Betatigungsglieds wird ein obergeordneter Punkt 
der Menustruktur durch die Dret,- bzw, LSngsbewegung des Betat,gu,.gsgl,eds e,n 
demgegenOber untergeordneter Punkt der Menustruktur ausgewShit. Im 
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(54) Title: DEVICE FOR CONTOOLLING A DISPLAY SCREEN 

(54) Bezeichnung: VORRICHTUNG ZUR STOUERUNG EINER BILDS CHIRM ANZEIGE 

(57) Abstract 

A device for controlling a display 
screen has an actuating element which can 
be rotated about a longitudinal axis and 
moved in the direction of the longitudinal 
axis, enabling a point of a menu struc- 
ture consisting of menu, partial menus, 
functions and/or function values to be se- 
lected, and which can be represented as 
an optically highlighted field in the dis- 
play screen. The actuating element has an 
initial position and can describe with rela- 
tion thereto an additional movement with 
two additional degrees of freedom. The 
additional movement of the actuating el- 
ement enables a field arranged in a mar- 
ginal region (2) of the display screen and 
associated with a point of the menu struc- 
ture to be selected. The rotary/longitudinal 
movement of the actuating element enables 
a subordinate field of the menu structure 
associated with the field arranged in the 
marginal region of the display screen to 
be selected in the central region (3) of the 
display screen enclosed by the marginal region. 
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(57) Zusammenfassung 

Bei einer Vorrichtung zur Steuemng einer Bildschirmanzeige mit einem Betatigungsglied, das um eine Langsachse drehbar und in 
Richtung der Langsachse bewegbar ist und wodurch ein Punkt einer aus Menu, Teilmeniis, Funktion und/oder Funktionswert bestehenden 
Menustniktur auswahlbar und als optisch hervorgehobenes Feld in der Bildschirmanzeige daretellbar ist, bei der das Betatigungsglied 
erne Ausgangslage besitzt und demgegeniiber eine Zusatzbewegung mit zwei zusatzlichen Freiheitsgraden durchfuhren kann, ist bei der 
Zusatzbewegung des Betatigungsglieds ein in einem Randbereich (2) der Bildschirmanzeige angeordnetes und einem Punkt der Meniistruktur 
zugeordnetes Feld auswahlbar, und mittels der Dreh-ZLSngsbewegung des Betatigungsglieds ist ein diesem Feld zugeordnetes, innerhalb 
der Menustniktur untergeordnetes Feld in dem vom Randbereich umfaSten Mittelbcreich (3) der Bildschirmanzeige auswmilbar. 
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Vorrichtung zur Steuerung einer Bildschirmanzetge 

5 

Die Erfindung bezieht sich auf eine Vorrichtung mit den Merkmalen des Oberbegriffs 
von Patentanspruch 1. 

10 Eine derartige Vorrichtung ist aus der EP 0 796 766 A2 bekannt. Dabei dient das 
Betatigungsgiied dazu, die Anzahl der Ebenen innerhalb der Menustruktur zu ver- 
groRern. Hierzu ist angegeben, die Ebenen uber den Umfang der Biidschirmanzeige 
verteilt anzuordnen. Eine derartige Vorrichtung bedeutet fur den Benutzer eine zu- 
satzliche Schwierigkeit, da er keine eindeutige Zuordnung zwischen der Zusatzbe- 

15 wegung und der Axial- bzw. Rotationsbewegung des Betatigungsgiieds vorfindet. 
Auch gestaltet sich der Ubergang zwischen den Ebenen und innerhaib der Menu- 
struktur fur ihn nicht eindeutig und fuhrt letztendlich zu einer erheblichen Ablenkung 
des Benutzers vom Verkehrsgeschehen, 

20 Der Erfindung liegt die Aufgabe zugrunde. eine Vorrichtung der eingangs genannten 
Art zu schaffen, bei der eine eindeutige Zuordnung zwischen der Biidschirmanzeige 
und den verschiedenen Bewegungsmdgiichkeiten des Betatigungsgiieds besteht. 

Die Erfindung lost diese Aufgabe durch die Merkmaie des Patentanspruchs 1 . 

25 

Nunmehr wird durch die Zusatzbewegung des Betatigungsgiieds ein ubergeordneter 
Punkt der Menustruktur durch die Dreh- bzw. Langsbewegung des Betatigungs- 
giieds ein demgegenuber untergeordneter Punkt der Menustruktur ausgewahlt. 
Konkret bedeutet dies beispielsweise, dali im Randbereich die mbgiichen Menus 
30 angeordnet sind, wahrend sich im Mittelbereich die zugehdrigen Untermenus bzw. 
Funktionen oder Funktionswerte befinden. Dabei sind im Mittelbereich nur die Teil- 
menus, die Funktion bzw. Funktionswerte angeordnet, die zu dem einen. vorzugs- 
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weise durch eine vorhergehende Zusatzbewegung des Betatigungsgiieds ausge- 
wahlten Menu gehoren. Im Randbereich befinden sich somit die innerhalb der Me- 
nustruktur auf gleicher Stufe stehenden Punkte, wahrend im Mittelbereich die hier- 
archisch niedrigeren Punkte zu finden sind. 

5 

Wie an sich aus der EP 0 796 766 A bekannt ist. kann das Betatigungsgiied sowohi 
eine Taumelbewegung als auch eine Parallelverschiebung ais Zusatzbewegung 
durchfuhren und dabei auch zwangsgefuhrt sein. Diese Bewegungsmoglichkeiten 
bzw. -einschrankungen verbessern den Umgang mit und die Bewegung innerhalb 

10 der Menustruktur. Ausgehend von der Mittellage als Ausgangslage des Betati- 
gungsgiieds ist die Zusatzbewegung gleichbedeutend mit einem Verlassen der 
durch die im Mittelbereich angegebenen Ebene der Menustruktur und das Aussu- 
chen der nachsthoheren Ebene. Die Zuruckbewegung in die Ausgangslage bedeutet 
wiederum den Ubergang von der soeben aufgesuchten hbheren Ebene in die dar- 

15 unter liegende nachste Ebene. Der Benutzer erhalt damit taktil ein Gefuhl dafur ver- 
mittelt, in welcher Weise er sich innerhalb der Menustruktur bewegt 

Eine Verbesserung der Erfindung beschaftigt sich mit dem Problem der Fehlbedie- 
nung, die bei einer derartigen Vorrichtung auftreten kann. Besonders dann, wenn 

20 eine Vielzahl von Punkten im Rahmenbereich der Bildschirmanzeige angeordnet ist, 
ist die Gefahr besonders groR, versehentlich nicht den gewunschten Punkt zu 
„treffen". Hinzu kommt die besondere Situation, in der die Auswahl dieser Punkte 
erfolgt. Haufig erfolgt die Hinwendung zur Bildschirmanzeige in Konkurrenz zum 
Verkehrsgeschehen. Die dafur zur Verfugung stehende Zeit ist i.d.R. knapp bemes- 

25 sen. 

Werden die im Rahmenb^eich angeordneten Punkte bei und/oder nach der Zu- 
satzbewegung optisch hergehoben und erst dann ausgewahit, wenn das Betati- 
gungsgiied sich zumindest annahernd wieder in der Ausgangslage befindet, wird die 
30 Gefahr einer Fehlauswahl weitgehend vermieden. 
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Mit Htlfe der Zusatzbewegung ist es zwar moglich, den im Rahmenbereich angeord- 
neten Punkt der Menustruktur optisch auszuwahien. Funktionell erfolgt die Auswahi 
jedoch erst dann. wenn das Betatigungsglied sich wieder in der Ausgangslage be- 
findet. Unter funktioneller Auswahi ist dabei bei Einsatz der Vorrichtung in einem 
Fahrzeug zur Steuerung verschiedener Menus, wie Navigationssystem. Bordcom- 
puter. Klimaaniage usw. und der Zuordnung der im Rahmenbereich angeordneten 
Punkte zu den einzelnen Menus folgendes zu verstehen: 



Durch die Zusatzbewegung des Betatigungsglieds wird der dem jeweiligen Menu 
10 zugeordnete Abschnitt des Randbereichs optisch hervorgehoben, beispieisweise 
gehighlightet. Die Auswahi und damit Ansteuerung des jeweiligen Menus, z. B. der 
Navigation erfolgt jedoch erst dann, wenn nach der optischen Hervorhebung des 
entsprechenden Randbereichabschnitts das Betatigungsglied wieder in die Aus- 
gangslage zuruckgefuhrt wird. Erst dann wird das Navigajionssystem anstelle des 
15 vorher auf den Bildschirm dargestellten Menus, beispieisweise fur den Bordcompu- 
ter. ausgewahit, Werden dabei die Teilmenus und/oder Funktionen und/oder Funkti- 
onswerte innerhalb der vom Rahmenbereich umgebenen Innenflache der Biidschir- 
manzeige als Punkte (z. B, in Form von rechteckigen Feldern) dargestellt, kbnnen 
nun diese Punkte vorzugsweise mit Hilfe des Betatigungsglieds aktiviert werden. 

20 

Der Ubergang zu einem anderen Menu erfordert wtederum die vorbereitende Aus- 
wahi des jeweiligen Randbereichabschnitts mit Hilfe des Betatigungsglieds, verbun- 
den mit der optischen Hervorhebung dieses Ausschnitts und der anschlielienden 
funktionellen Auswahi durch Ruckfuhren des Betatigungsglieds in die Ausgangsla- 
25 ge. 

Anhand der Zeichnung ist die Erfindung weiter eriautert. Es zeigt 

Fig. 1 eine Biidschirmanzeige. die im Rahmen der erfindungsgemafJen Vorrichtung 
30 verwendet wird, 
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Fig. 2 die Bildschirmanzeige von Fig, 1 bei Ansteuerung durch ein aus der EP 0 
796 766 A bekanntes Betatigungsglied. 

Fig. 3 eine Weiterbildung der Vorrichtung von Fig. 1 und 2 und 

5 

Fig. 4 und 5 die Wirkungsweise der Vorrichtung von Fig. 3. 

In Fig. 1 ist die im Rainmen der Erfindung vorgesehene Bildschirmanzeige prinzipieil 
dargestellt. Innerhalb einer Anzeigeflache 1. die gleich der Bildflache eines Bild- 

10 schirms ist, befindet sich ein Randbereich 2 und ein Mittelbereich 3. Im Randbereich 
2 sind an acht Stellen. die bezogen auf den Mittelpunkt der Anzeigeflache 1 in etwa 
jeweils urn 45° gegeneinander versetzt sind, acht Symbole fur die mit einem nicht 
dargesteilten Betatigungsglied anzusteuemden Menus dargestellt. Es handelt sich 
dabei um die Menus Audio, Navigation, Fernsehen (TV), Stand-Klima usw.. 

15 „Einstellung" bedeutet, den insgesamt einstelibaren Funktionen bzw. Funktionswerte 
zugeordnet, „Ofr steht fur die Moglichkeit. die Bildschirmanzeige auszuschaiten. 

Durch eine Zusatzbewegung (Taumel- bzw. Parallelbewegung) des Betatigungs- 
glieds (nicht dargestellt) kann einer der acht Menupunkte ausgewahit werden. Hier- 
20 zu ist das Betatigungsglied in einer durch einen Pfeil 4 symbolisierten Richtung zu 
bewegen. Hier wird der Menupunkt Audio ausgewahit. Diese Auswahl geschieht 
durch Bewegen des Betatigungsglieds in der durch den Pfeil 4 angegebenen Rich- 
tung um eine Strecke bzw. einen Winkel, der ein vorgegebenes Ma(i ubersteigt. 
Damit ist der Menupunkt „Audio" ausgewahit. 



Nach Loslassen geht das Betatigungsglied unter der Wirkung einer Ruckstellkraft 
wieder in die Ruhelage zurfeick. Gleichzeitig erscheinen innerhalb der Anzeigeflache 
1 die dem Menupunkt 4 zugeordneten hierarchiemafSig untergeordneten 
Menupunkte. Es handelt sich dabei um die innerhalb eines nicht dargesteilten 
Rundfunkgerats fest programmierten Sendestationen. Diese werden im Mittelbe- 
reich angezeigt. Dies ist in Fig. 2 gezeigt. Eines der Symbole, hier ..Antenne" ist op- 
tisch hervorgehoben. Es handelt sich dabei um das Symbol des Senders, der zuletzt 



25 
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ausgewahit wurde. Nimmt der Fahrzeugbenutzer keine weiteren Einsteliungen vor. 
so bleibt dieser Sender eingestellt. 

Mochte er hingegen den Sender andern, so dreht er hierzu das Betatigungsglied um 
seine Langsachse, Es handelt sich dabei um einen Dreh-Druck-Knopf, wie er aus 
der EP 0 366 132 B1 bekannt ist. und auch in der EP 0 796 766 A beschrieben ist. 
Dabei wird eines der den anderen Sendern zugeordneten Symbole, z. B, Klassik 
optisch anstelle von Antenne hervorgehoben. Die Umschaltung auf diesen Sender 
erfolgt, indem der Dreh-Druck-Knopf in Langsrichtung bewegt wird. Unter der Wir- 
kung einer Ruckstellfeder geht er anschliefiend wieder in seine Ausgangslage zu- 



Soli nun auf eine Funktion eines anderen Menus, beispieisweise des Menus 
BC=Bordcomputer umgeschaltet werden, ist hierzu der Dreh-Druck-Knopf in der 
durch den Pfeil BC angegebenen Richtung zu verschwenken bzw. parallel zu ver- 
schieben. Mit dem Verschwenken einher geht die Anzeige der verschiedenen Funk- 
tionen des Bordcomputers im Mittelbereich anstelle der in Fig. 2 dargesteliten Sen- 
dersymbole. Eine Bordcomputer-Funktion, z. B. Reichweite kann standardmafiig 
Oder nach vorbereitender Axialbewegung des Dreh-Druck-Knopfs hervorgehoben 
sein. Die Auswahl dieser Funktion, d. h. die Anzeige der tatsachllchen Reichweite 
des Fahrzeugs, kann dann erfolgen, indem der Dreh-Druck-Knopf gegebenenfalls 
nochmals axial bewegt wird. im Mittelbereich erscheint beispieisweise neben dem 
optisch hervorgehobenen Symbol ..Reichweite" der Wert der Reichweite, z, B. 225 
km, 

Unmittelbar danach kann auf das Menu Navigation umgeschaltet werden. indem die 
Zusatzbewegung des Dreh^Druck-Knopfs in Richtung des Pfeils ..Navigation" durch- 
gefiihrt wird. Im Mittelbereich erscheinen die einzeinen Navlgationsparameter. 



30 



Auf diese Weise ist es mdgiich, rasch und sicher zwischen den einzeinen Menus, 
den Teilmenus, Funktionen und Funktionswerte umzuschalten. 
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Die anhand von Fig. 3 ff. dargestellte Weiterbildung der Erfindung besteht darin. 
beim translatorischen Verschieben des Betatigungsgiieds nicht sofort einen Funkti- 
onswechse! auszulosen, sondern zunachst nur eine Hervorhebung, 2. B. ein 
"highlighting", des gewahlten Hauptmenupunkts vorzunehmen. Dadurch ist zu die- 
5 sem Zeitpunkt noch eine Korrektur durch den Bedienenden moglich. Die eigentliche 
Funktionsauslosung erfolgt erst mit dem Loslassen des federzentrierten Betati- 
gungsgiieds. 

Der Nutzer mochte den Funktionsbereich "Audio" aufrufen. Im Beispiel von Fig. 3 
10 wurde statt "Audio" versehentlich der Menupunkt "Navigation" gewahlt. Wurde jetzt 
sofort das zugehorige Untermenii aufgerufen, befande sich der Bediener im Navi- 
gationsmenu. 

Urn dies zu Vermeiden, erfolgt der Aufruf des jeweiligen Untermenus nicht sofort bei 
15 der Auswahl, sondern erst beim Loslassen des Betatigungsgiieds. Im Beispiel er- 
kennt der Nutzer an der Hervorhebung von "Navigation", daS der falsche Menupunkt 
gewahlt ist und kann das Betatigungsglied nach links auf den gewunschten 
Menupunkt "Audio" schieben. Dies zeigt Fig. 4. Auch jetzt ist die zugehorige Funkti- 
on „Menu'* Audio noch nicht aktiviert. 



Erst durch Loslassen des federzentrierten Betatigungsgiieds erfolgt der Aufruf des 
gewunschten Untermenus "Audio", wie in Fig. 5 gezeigt. Fig. 5 ist unterscheidet sich 
gegenuber der in Fig. 2 dargestellten Situation nur unsoweit, als nunmehr die Be- 
schriftung des aktivierten Menus im Rahmen optisch hervorgehoben bleibt. 



20 
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Patentanspriiche 



1. Vorrichtung zur Steuerung einer Bildschirmanzeige mit einem Betatigungsgiied, 
das urn eine Langsachse drehbar und in Richtung der Langsachse bewegbar ist 
und wodurch ein Punkt einer aus Menu, Teiimenus, Funktion und/oder Funkti- 
onswert bestehenden IVIeniistruktur auswahlbar und ais optisch hervorgehobenes 
Feid in der Bildschirmanzeige darstellbar ist. bei der das Betatigungsgiied eine 
Ausgangslage besitzt und demgegenuber eine Zusatzbewegung mit zwei zusatz- 
lichen Freilieitsgraden durchfuhren kann, dadurch gekennzeiclnnet, daft bei 
der Zusatzbewegung des Betatigungsgiieds ein in einem Randbereich der Bild- 
schimnanzeige angeordnetes und einem Punkt der Menustruktur zugeordnetes 
Feid auswahlbar ist, und daS mittels der Dreh-ZLSngsbewegung des Betatigungs- 
giieds ein diesem Feid zugeordnetes, innerhaib der Menustruktur untergeordne- 
tes Feid in dem vom Randbereich umfaBten Mittelbereich der Bildschirmanzeige 
auswahlbar ist. 

2. Vorrichtung nach Anspruch 1, dadurch gekennzeichnet, daG das Betati- 
gungsgiied eine Taumelbewegung ais Zusatzbewegung durchfuhren kann. 

3. Vorrichtung nach Anspruch 1, dadurch gekennzeichnet. dalS das Betati- 
gungsgiied eine Parallelverschiebung ais Zusatzbewegung durchfQhren kann. 

4. Von-ichtung nach Ansprtich 2 Oder 3 , dadurch gekennzeichnet, da(i das 
Betatigungsgiied fur die Zusatzbewegung zwangsgefuhrt ist. 

5. Vorrichtung nach einem der Anspruche 1 bis 4, dadurch gekennzeichnet, 
dali das Betatigungsgiied eine reversible Zusatzbewegung durchfuhrt. 



wo 99/27435 



PCT/EP98/0757I 



8 

6. Vorrichtung nach Anspruch 5, dadurch gekennzeichnet . daa das Betati- 
gungsglied die Zusatzbewegung gegen die Wirkung einer Feder ausfuhrt. 

7. Vorrichtung nach einem der Anspruche 1 bis 6. dadurch gekennzeichnet, 
dad die im Rahmenbereich angeordneten Punkte bei und/oder nach der Zusatz- 
bewegung optisch hergehoben sind und erst dann ausgewahlt sind, wenn das 
Betatigungsglied sich zumindest annahernd wieder in der Ausgangslage befindet. 

8. Vorrichtung nach einem der Anspruche 1 bis 7 dadurch gekennzeichnet, 
dafi bei Auswahi des Feldes im Randbereich die zugeordneten auswahlbaren 
Felder im Mittelbereich seibstatig angezeigt sind. 



wo 99/27435 



PCT/EP98/07S71 



1/3 
3 



: Audio 




DSP 




K-BCsx 



^EinsiellungN^ 



Audio 





IN 

Stand- 



kl 



ma 




FI6.1 



Beldiigung 

Durch Neigen wird das 
Audiomenii oufgerufen 




Bayern 3 
B5 Akiuell 



Antenne 




Standi 
Klimo 



Charivari 
Klassik 




FIG.2 

Bildschirm 



§ j;:)Hauptmenijpunkte 



Navigation 
Audio 4 TV 



DSP ^ 




^ Stand- 
Klima 



BC i Off 
Einstellung 

ERSATZBLATT (REGEL 26) 



Dreh-Driick-Knopf mit 
joystick -dhnlicher 
Zusatzfunktionalitdt 



wo 99/27435 



PCT/EP98/07571 




ERSATZBLATT(REGEL26) 



wo 99/27435 



PCT/EP98/07571 





INTERNA^JlVAL SEARCH REPORT 



A-«9!-^l^"''*^*'^'<^N OF SUBJECT MATTER 

IPC 6 G06F3/033 



According to Internattonal Patent Classification (IPC) or to both national ciassification and IPC 



tional Application No 

PCT/EP 98/07571 



B. FIELDS SEARCHED 



Minimum documentation searcned (classification system followed by classification symbols) 

IPC 6 G06F 



Documentation searched other than minimum documentation to the extent that such documents are included in the fields searched 



Electronic data base consulted during the international search (name of data base and. where practical, search terms used) 



C. DOCUMENTS CONSIDERED TO BE RELEVANT 



Category * 



Citation oi document, with indication, where appropriate, of the relevant passages 



EP 0 796 766 A (PHILIPS PATENTVERWALTUNG 

;PHILIPS ELECTRONICS NV (NL)) 

24 September 1997 

cited in the application 

see the whole document 

WO 96 30822 A ( FOREST DONALD K ) 
3 October 1996 

see page 42, line 18 - line 34 
see page 86, line 15 - line 25 
see figure 19 
see figures 52,53,64,65 

US 4 794 388 A (MATTHEWS HENRY G) 
27 December 1988 

see column 11, line 45 - column 12, line 
37; claims 1-3; figures 1,4 

-/-- 



Relevant to claim No. 



2-6 

1 

7 

1-6 



X I further documents are listed in the continuation of box C. 



Patent family members are listed in annex. 



Special categories of cited documents : 

"A" document defining the general state of the art which is not 
considered to be ot particular relevance 

'E- earlier document but published on or atter the internatiohal 
filing date 

"L" document which may throw doubts on priority ciaim(s) or 
whtch IS cjted to establish the publication date of another 
citation or other special reason (as specified) 

'O" document referring to an oral disclosure, use. exhibition or 
other means 

'P" document published prior to the international filing date but 

lator than tha t-xri^riu, ^i*^ i ^ 



later document published after the international filing date 
or priority date and not in conflict with the application but 
cited to understand the principle or theory undertying the 
invention 

document of particular relevance; the claimed invention 
cannot be considered novel or cannot be considered to 
involve an inventive step when the document is taken alone 

document of particular relevance: the claimed invention 
cannot be considered to involve an inventive step when the 
document is connbined with one or more other such docu- 
ments, such combination being obvious to a person skilled 
in the art. 



Date of the actual completion of the international search 

30 March 1999 


Date of mailing of the international search report 

08/04/1999 


Name and mailing address of the ISA 

European Patent Office, P.B. 5818 Patentlaan 2 
NL - 2280 HV Rijswijk 
Tel. <+31-70) 340-2040. Tx. 31 651 epo nl. 
Fax: (+31-70) 340-3016 


Authorized officer 

Durand, J 



page 1 of 2 



INTERN, 



itb 



NAL SEARCH REPORT 



itional Apptication No 

PCT/EP 98/07571 



C.(Continuatton) DOCUMENTS CONSIDERED TO BE RELEVANT 



Category ■ 



Citation of document, with indication.where appropnate, ot the relevant passages 



Relevant to claim No. 



EP 0 498 082 A (KONINKL PHILIPS 

ELECTRONICS NV) 12 August 1992 

see column 6, line 22 - line 35; figures 

3,7 

US 5 297 253 A (MEISEL LESLIE M) 
22 March 1994 

see column 7, line 9 - line 37; figures 

PATENT ABSTRACTS OF JAPAN 

vol. 097, no. 009, 30 September 1997 

& JP 09 114902 A (HITACHI LTD;HITACHI 

SEIBU SOFTWARE LTD), 2 May 1997 

see abstract 

"ALGORITHM FOR DECREASING THE ERROR RATE 
OF DATA ENTERED ON A TOUCH-SENSITIVE 
TERMINAL" 

IBM TECHNICAL DISCLOSURE BULLETIN, 
vol. 33, no. lOA, 1 March 1991, pages 
223-227, XPOOOl 10024 
see page 224 

EP 0 623 870 A (IBM) 9 November 1994 
see the whole document 

EP 0 366 132 A (BAYERISCHE MOTOREN WERKE 

AG) 2 May 1990 

cited in the application 

see the whole document 



1,7 



8 

1,7 



Form PCT/ISA/210 (contmuaiion ot second sheet) (Juty 1992) 



page 2 of 



2 



INTERNi^^^NAL SEARCH REPORT 

Information on patent family members 



m 



Patent document 
cited in search report 



Publication 
date 



tional Application No 

PCT/EP 98/07571 



Patent family 
member(s) 



Publication 
date 



EP 


0796766 


A 


24- 


-09- 


-1997 


DE 


19610344 


A 


25-09-1997 














JP 


10020950 


A 


23-01-1998 














1 IC 




A 
A 


16-03-1999 


WO 


96'^OR?? 


A 
rt 




-10- 




Al i 


9 1 Q 1 HOC 


A 

A 


16-10-1996 














DE 


19581933 


T 


16-04-1998 














GB 


2314186 


A 


17-12-1997 


US 


4794388 


A 


27- 


-12- 


■1988 


NONE 








EP 


0498082 


A 


1 9- 
1 c 








A01 907^^/l 


n 
U 


10-06-1998 
















oy i^yjo4 


T 
1 


22-10-1998 














JP 


4317119 


A 


09-11-1992 














us 


5828360 


A 


27-10-1998 


US 




A 
r\ 


CC 


U J 












FP 




A 

M 




-11- 


1 QQ/1 


r A 

LA 




A 

A 


05-11-1994 














IP 

Or 


o o >! Q 


A 

A 


A'^ lO irtrtJl 

02-12-1994 


















A 


24-09-1996 














IK 


booyooo 


A 

A 


1 O 11 1 AATf 

18-11-1997 


EP 


0366132 


A 


02- 


■05- 


1990 


DE 


3836555 


A 


10-05-1990 














DE 


4001062 


A 


18-07-1991 














DE- 


58908735 


Q 


19-01-1995 














ES 


2064418 


T 


01-02-1995 














JP 


2187814 


A 


24-07-1990 














JP 


2840332 


B 


24-12-1998 














US 


5270689 


A 


14-12-1993 



Form PCT/ISA/210 (patent ramily annex) (July 1992) 



international: 



E^^UECHERCHENBERICHT 



A. KLASSIFIZIERUNG DES ANMELOUNGSGEGENSTANDES 

IPK 6 G06F3/033 



Nach der Internationalen PatentklassifikaUon (IPK) oder nach der nationalen Klassilikalion und der IPK 



tionaies Aktenzeichen 

PCT/EP 98/07571 



B. RECHERCHIERTE GEBIETE 



Recherchrerter Mindestprufstoft (Klassifikationssystem und Klassitikationssvmbole ) 

IPK 6 G06F 



Recherchierte aber nicht zum Mindestprufstoft gehorend© Verotfentlichungen. soweit diese unter die racherchierten Gebiete fallen 



Wahrend der internationalen Recherche konsultierte elektronische Oatenbank (Name der Datenbank und evtl. verwendete Suchbegrlffe) 



C. ALS WESENTLICH ANGESEHENE UNTERLAGEN 



Kategorie" 


Bezelchnung der Veroftentlichung, sowett ertordertich unter Angabe der in Betracht kommenden Telle 


Betr. Anspruch Nr. 


Y 


EP 0 796 766 A (PHILIPS PATENTVERWALTUNG 


1 




;PHILIPS ELECTRONICS NV (NL)) 






24. September 1997 






in der Anmeldung erwahnt 




A 


siehe das ganze Ookument 


2-6 


Y 


WO 96 30822 A (FOREST DONALD K) 


1 




3. Oktober 1996 




A 


siehe Seite 42, Zeile 18 - Zeile 34 


7 




siehe Seite 86, Zeile 15 - Zeile 25 






siehe Abbildung 19 






siehe Abbildungen 52,53,64,65 




A 


US 4 794 388 A (MATTHEWS HENRY G) 


1-6 




27. Dezember 1988 






siehe Spalte 11, Zeile 45 - Spalte 12, 






Zeile 37; Anspriiche 1-3; Abbildungen 1,4 






-/-- 





Weiter© Veroftentlichungen sind der Fortsetzung von Feld C zu 
entnehmen 



ID 



Siehe An hang Patentfamili© 



' Besondere Kategorien von angegebenen Verotfentlichungen 

'A" Veroffentlichung, die den allgemeinen Stand der Technik definiert. 
aber nicht als besonders bedeutsam anzusehen ist 

E" alteres Dokument, das jedoch erst am Oder nach denri Internationalen 
Anmeldedatum veroffentllcht worden ist ' 

L" Veroffentlichung. die geeignet ist, einen Prioritatsanspruch zweifelhaft er- 
scheinen zu lassen. oder durch die das Veroftentltchungsdatum einer 
anderen im Recherchenbericht genannten Veroffentlichung belegt warden 
soil Oder die aus einem anderen besonderen Grund angegeben ist (wie 
ausgetuhrt) 

O" Veroffentlichung, die sich auf eine mundliche Offenbarung. 

eIne Benutzung, ©ine Ausstellung oder ander© MaBnahmen bezieht 
P" Veroffentlichung, die vor dem internationalen Anmeldedatum, aber nach 
dem beanspruchten Prioritatsdatum veroffentllcht worden ist 



T" Spate re Veroffentlichung, die nach dem internationalen Anmeldedatum 
Oder dem Prioritatsdatum veroffentlicht worden ist und mit der 
Anmeldung nicht kollldiert, sondern nur zum Verstandnis des der 
Erfindung zugrundeliegenden Prinzips oder der ihr zugrundeliegenden 
Theorie angegeben ist 

"X" Veroffentlichung von besonderer Bedeutung; die beansprucht© Erfindung 
kann allein aufgrund dieser Veroffentlichung nicht als neu Oder auf 
ertinderischer Tatigkeit beruhend betrachtet werden 

"Y" Veroffentlichung von besonderer Bedeutung; di© beansprucht© Erfindung 
kann nicht als auf erfindenscher Tatigkeit beruhend betrachtet 
werden, wenn die Veroffentlichung mit einer oder mehreren anderen 
Veroftentlichungen dieser Kategorie in Verbindung gebracht wird und 
diese Verbindung fur einen Fachmann naheliegend ist 

"&" Veroffentlichung, die Mitglied derselben Patentfamili© ist 



Datum des Abschiusses der internationalen Recherche 

30. Marz 1999 


Absendedatum des internationalen Recherche nberichts 

08/04/1999 


Nam© und Postanschritt der internationalen Recherchenbehorde 
Europaisches Patentamt. P.B. 5818 Patentlaan 2 
NL - 2280 HV Rijswiik 
Tel. (+31-70) 340-2040. Tx. 31 651 epo nl, 
Fax: (+31-70) 340-3016 


Bevollmachtlgter Bediensteter 

Durand, J 



Seite 1 von 2 



INTERNATION. 



RECHERCHENBERICHT 



C.(Fortsetzung) ALS WESENTLICH ANGESEHENE UNTERLAGEN 



f 

I PC 



Itionales Aktenzeichen 

PCT/EP 98/07571 



Kategone^ 



Bezeichnung der Veroffentlichung, soweit ertorderiich unter Angab© der in Betracht kommenden Teiie 



EP 0 498 082 A (KONINKL PHILIPS 
ELECTRONICS NV) 12. August 1992 
siehe Spalte 6, Zeile 22 - Zeile 35; 
Abbildungen 3,7 

US 5 297 253 A (MEISEL LESLIE M) 
22. Marz 1994 

siehe Spalte 7, Zeile 9 - Zeile 37; 
Abbildungen 

PATENT ABSTRACTS OF JAPAN 
vol. 097, no. 009, 30. September 1997 
& JP 09 114902 A (HITACHI LTD; HITACHI 
SEIBU SOFTWARE LTD), 2. Mai 1997 
siehe Zusanmenfassung 

"ALGORITHM FOR DECREASING THE ERROR RATE 
OF DATA ENTERED ON A TOUCH-SENSITIVE 
TERMINAL" 

IBM TECHNICAL DISCLOSURE BULLETIN, 
Bd. 33, Nr. lOA, 1. Marz 1991, Seiten 
223-227, XP000110024 
siehe Seite 224 

EP 0 623 870 A (IBM) 9. November 1994 
siehe das ganze Dokument 

EP 0 366 132 A (BAYERISCHE MOTOREN WERKE 

AG) 2. Mai 1990 

in der Anmeldung erwahnt 

siehe das ganze Dokument 



Betr. Anspruch Nr. 



1,7 



8 

1,7 



Formblati PCT/lSA/210 (Forteettung von Blalt 2) (Juli 1992) 



Seite 2 von 2 



internahonale^^jecherchenbericht 

Angaben zu Veroffentiichuirtjen^ie zur seiben Patentfamitie gehoren 



ionales Aktenzeichen 

PCT/EP 98/07571 



im Recherchenbehcht 


Datum der 




Mitglied(er) der 




Datum der 


angefuhrtes Patentdokument 


Veroffentlichung 




Patentfamilie 




Verflffentlichung 


EP 0796766 A 


24-09-1997 


DE 


19610344 


A 


25-09-1997 






JP 


10020950 


A 


23-01-1998 






US 


5883346 


A 


16-03-1999 


WO 9630822 A 


03-10-1996 


AU 


2191095 


A 


16-10-1996 






DE 


19581933 


T 


16-04-1998 






GB 


2314186 


A 


17-12-1997 



US 4794388 A 27-12-1988 KEINE 



EP 


0498082 


A 


12- 


-08- 


■1992 


DE 69129364 


D 


10-06-1998 














DE 69129364 


T 


22-10-1998 














JP 4317119 


A 


09-11-1992 














US 5828360 


A 


27-10-1998 


US 


5297253 


A 


22- 


•03- 


1994 


KEINE 






EP 


0623870 


A 


09- 


■11- 


1994 


CA 2095452 


A 


05-11-1994 














JP 6332648 


A 


02-12-1994 














US 5559945 


A 


24-09-1996 














US 5689668 


A 


18-11-1997 



EP 0366132 A 02-05-1990 DE 3836555 A 10-05-1990 

DE 4001062 A 18-07-1991 

. . - -- - DE 58908735 D 19-01-1995 

ES 2064418 T 01-02-1995 

JP 2187814 A 24-07-1990 

JP 2840332 B 24-12-1998 

US 5270689 A 14-12-1993 



Formblatt PCT/ISA/210 (Anhang Patentfamtlje)(Juli 1992) 



0) 



i 




PCr/US95/03591 



WO 96/30822 



10 



15 



20 



25 



displayed on the selectable regions cannot be seen in a glance by many operators. However, when 
the displayed menu options are displayed on the indicating regions, for example, as shown in 
Figure 52, the displayed menu options can be seen in a glance, facilitating searching of the menu 
by the operator for his intended menu option. Since the location of each selectable region is 
indicated by the location of the associated indicating region, the operator may point to the intended 
selectable region without searching it out or pausing to verify that the menu option associated with 
a selectable region is the menu option he desires. The frequent operator of such a menu interface 
may habituate the process of selecting an intended menu option so he can focus his attention on 
another task while selecting the option, for example, planning his next interaction with the menu 
interface. 

Figure 53 depicts a display and structures in accord with an another embodiment of the Location 
Indication aspect of the invention. Selectable regions (4604), (4608), (4612), (4616), (4620), 
(4624), (4628) and (4632) are located adjacent the display (2! 12) and associated respectively with 
menu options "so", "say", "she", "suffixes", "said", "show", "some" and "words". Each selectable 
region is unbounded on the side furthest from and parallel to the edge of the display . Indicating 
regions (4351), (4353), (4355), (4357), (4367), (4361), (4363) and (4365) are associated 
respectively with menu options "so", "say", "she", "suffixes", "said", "show", "some" and "words". 
In accord with the Location Indication aspect of the invention, the location of an indicating region 
indicates the location of the selectable region associated with the menu option associated with the 
indicating region. 

The large size of the selectable regions outside the display, for example, as shown in Figure 53, 
facilitate selection by individuals with impaired fme motor control while the indicating regions 
indicate the location of each associated selectable region. 

Yet another embodiment in accord with the Location Indication aspect of the invention is shown in 
Figures 54 and 55. Figure 54 depicts a display (2112) having thereon eight selectable regions, each 
associated respectively with a menu option. As shown in the Figure, selectable region (391 1) is 
associated with menu option "aeiou". selectable region (3910) with menu option "gqjyvwxz", 
selectable region (3908) with menu option "setup", selectable region (3907) with menu option 
undo, selectable region (3905) with menu option "control", selectable region (3904) with menu 
option "words", selectable region (3902) with menu option "nsnn<space>tcp" and selectable region 
(3901) with menu option "bdfhkl". Menu option "aeiou" is associated w ith a submenu which 
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includes submenu options "a", "e", "i", "o" and "u'\ displayed on indicating regions (3930), (3928), 
(3926), (3924) and (3922) respectively. The background pattern of indicating region (3930) 
matches the background panem of selectable region (3908), indicating that submenu option "a" 
will be associated with selectable region (3908) following selection of menu option "aeiou". 
5 Similarly, the background pattern of indicating region (3928) matches the background pattern of 
selectable region (3910), indicating that submenu option "e" will be associated with selectable 
region (3910) following selection of menu option "aeiou". Alternatively, the indication may be 
made by the size, shape, hue. brightness, contrast, dithering, fill, blinking, hatching or panem of the 
indicating region or any part thereof, including either of the foreground and background of the 
10 indicating region. 

When the operator selects selectable region (391 1), the display (2112) changes to that shown in 
Figure 55. Now, selectable region (3910) is associated with submenu option "e". selectable region 
(3908) with submenu option "a", selectable region (3904) with submenu option "u". selectable 
15 region (3902) with submenu option "o" and selectable region (3901 ) with submenu option '*i'*. In 
accord with the Location Indication aspect of the invention, the appearance of an indicating region 
indicates the location of the selectable regions associated with the submenu option associated with 
the indicating region. 

20 The embodiment shown in Figures 54 and 55 illustrates how the Location Indication aspect of the 
invention speeds selection of a submenu option of a menu hierarchy. The operator, by observing 
the background pattern of the submenu option within the displayed menu option, may determine 
which selectable region he should next dwell on. The operator may make this determination prior 
to selection of the menu option and need not wait for the submenu options to be displayed on their 

25 associated selectable regions. In accordance with the Location Indication aspect of the invention, 
the operator may select or spell out words more quickly than with conventional automated menu 
hierarchy. Assuming the display (2112) is part of computer system, these words may be input to an 
application program, and , if the computer system is coupled to a speech synthesizer, these words 
may be spoken. 



30 



Figure 56 depicts another apparatus in accord with the Location Indication aspect of the invention. 
An example of the operation of this apparatus will now be described. Figure 56 depicts display 
area (4770) adjacent to which are located selectable regions (4701 ), (4703), (4705), (4707). (4709). 
(471 1 ), (4713) and (4715). Not shown in the Figure are three sets of eight menu options. At 
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different times during the operation of the apparatus shown in Figure 56, for example at one second 
intervals, the plurality of selectable regions is associated with a different set of eight menu options 
such that, for a one second period, each selectable region is associated respectively with one menu 
option of the associated set of menu options. On the display area are 24 indicators, each associated 
5 respectively w ith one of the menu options. Each indicator indicates when a selectable region is 

associated with the menu option associated with the indicator. Thus, each of the indicators (472 1 ), 
(4723), (4725), (4727), (4729), (473 1 ), (4733) and (4735) is associated respectively with a menu 
option which may be in turn associated respectively, for example, during a first one second period, 
with one of the selectable regions. The operator may select a desired one of the 24 menu options 
10 by selecting the associated selectable region during the period when the desired menu option is 
associated with the associated selectable region. 

Alternatively, the apparatus shown in Figure 56 may require two successive selection events, the 
first selection event to select a set of eight menu options or to select a set of three menu options, the 
15 set being associated with one selectable region, and the second selection event to select one of the 
selected set. For example, the first selection event may select the set of menu options associated 
with indicators (4773), (4753) and (4733), and the second selection event may select one menu 
option from this set. 



20 Still another apparatus in accord with the Location Indication aspect of the invention is shown in 
Figure 57. Figure 57 depicts display area (4770) outside of which are located selectable regions 
(5001), (5003), (5005), (5007), (5009), (501 1), (5013), (501 5), (5021), (5023), (5025), (5027), 
(5029), (503 1 ), (5033) and (5035). The location of each selectable region is indicated by an 
indicating region on the display area, (5041), (5043), (5045), (5047), (5049), (5051), (5053), 

25 (5055), (5061), (5063), (5065), (5067), (5069), (5071), (5073) and (5075), respectively. 

The Location Indication aspect of the invention will now be described from an implementation 
perspective with reference to Figure 52. Preferably, the Location Indication aspect is implemented 
by modifications to the access program (1206) described in the detailed description of the 
30 Perimeter Menu aspect of the invention. The modifications required for the Location Indication 
aspect of the invention are: (1) at initialization time: (a) create twelve child edit windows, each 
corresponding to one of the indicating regions shown in Figure 52, each of the class "edit", each 
having the style (WS_CmLD | WS_BORDER | WS_DISABLED | ES_IVIULTIL1NE | 
AHgnmentStyle) where "1" represents a logical OR operation and where AlignmentStyle equals 
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ES_RJGHT for child windows having right justified text and ES_LEFT for child windows having 
left justified text, as shown in Figure 52; (b) move the child edit windows so they are located in or 
near the center of the display; (c) display the child edit windows; and (d) initialize an array which 
maps each child edit window, for example, by an index between 0 and 1 1 inclusive, to an element 
5 of the aLabel array; and (2) modify the code implementing ST SELECTED state processing, so 
that after changing the menu options associated with various selectable regions (by changing the 
pLabel element of the data set associated with each of the state machines to point to the menu 
option in the aLabel array to be associated with that state machine), the array mapping each child 
window to an element of the aLabel array is used to access the text of the menu option and the text 
10 of each child edit window is set accordingly. In the preferred embodiment, the six menu options 

displayed in indicating regions (4371), (4374), (4375), (4381), (4383) and (4385) are present in the 
aLabel array when Figure 52 is shown, but are not associated with selectable regions until the 
operator selects menu option "words". 



15 I. Sound Match 



The preferred embodiment of the Sound Match aspect of the invention will now be described in 
detail from a ftinctional perspective using an example depicted in Figure 58. Figure 58 shows a 
display (21 12) of a computer system (2116) having thereon six regions or sound indicators, (5801 ), 

20 (5803), (5805), (5807), (5809), and (58 11), each associated with a menu option, "ypgqj/\ "Idhbfk", 
"wizen", "words", "vort x", and "sumac", respectively. Each menu option is displayed on its 
associated sound indicator. Each sound indicator has a distinct hue. For example, sound indicator 
(5801) may be green, (5803) white, (5805) blue, (5807) red, (5809) orange and (581 1) grey. Each 
of these sound indicators indicates a sound the operator is able to consistently produce, for 

25 example, the vowel sound e as it sounds in green, i as it sounds in white, u as it sounds in blue, e as 
it sounds in red. o as it sounds in orange and e it sounds in grey. Assuming the operator says o as it 
sounds in orange, "vort x", the menu option associated with the orange sound indicator (5809) is 
selected and the display is changed to that shown in Figure 59. In the preferred embodiment, 
submenu options space, "x", "r", "t", "v" and "o" are now associated with each of the sound 

30 indicators (5901), (5903), (5905), (5907), (5909), and (591 1 ) respectively. Assuming the operator 
says e as in green, submenu option space is selected and a space is input to the application program 
(6107) whose output appears in region (5850). 

Figure 60 illustrates a display and structures in accordance with an alternative embodiment of the 
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Sound Match aspect of the invention. Figure 60 depicts a display (2 1 1 2) of a computer system 
(2 ! 1 6). Adjacent the display (2 1 1 2) is a static display (6252) having thereon eight sound indicators 
(6204), (6208), (6212). (6216), (6220), (6224), (6228). and (6232). Each of the eight sound 
indicators is a symbol in a phonetic character set graphically representing a sound. Alternatively, 
the sound indicators may include a picture of an object, for example, a tree, a house, a boy, or a 
map of a country, or may include a shape, for example, a square, a circle, or a triangle, or may 
include a hue indicator, a pitch indicator, a volume indicator, a sound duration indicator, a change 
in pitch indicator, or a change in volume indicator. Eight menu options, "vort<space>x'*, "sumac", 
"wizen"', undo, "words", "talk". "Idhbfk" and "ypgqj.". are displayed on the display (21 12). each 
adjacent and associated respectively with one of the sound indicators, the eight menu options 
together circumscribing region (6250) on the display. The operator may select any one of the menu 
options by producing the sound indicated by the sound indicator associated with the menu option. 
In response to a menu selection, submenu options of the selected menu option may be displayed 
and associated respectively with a sound indicator. 

An operator with impaired speech but who is able to consistently produce a relatively.small number 
of sounds distinguishable by conventional speech recognition means may, in accord with the Sound 
Match aspect of the invention, select from among the small number of menu options by using the 
sounds he can produce. Assuming the display (2112) is part of computer system, the menu option 
may represent inputs to an application program, and, if the computer system is coupled to a speech 
synthesizer, the menu option may represent words to be spoken. 

Figure 61 illustrates a display and structures in accordance with an alternative embodiment of the 
Sound Match aspect of the invention. Figure 61 depicts a display (21 12) of a computer system 
(2116) having thereon six square sound indicators, (6401), (6403), (6405), (641 1), (6413), and 
(6415) arranged in two columns of three sound indicators. Each sound indicator abuts a sound 
indicator in the other column, the sound indicator located above it, if any, and the sound indicator 
located below it, if any. Assume that the operator is entering Chinese, that he uses a keyboard to 
enter a phonetic unit and intonation according to the Pin Yin coding method for the Chinese 
language, and that he enters the distinct sound "fCi", each of the sound indicators shown in Figure 
61 is associated respectively with a menu option, each of the plurality of menu options having a 
common characteristic. In this example, the menu options are homophones and the common 
characteristic is a phonetic unit and intonation but may be a phonetic unit alone, an intonation 
alone, a stroke used to draw an ideograph, a number of horizontal strokes, a number of vertical 
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strokes, a number of total strokes* a stroke order, a radical, a part of speech, an ideograph, a kana, a 
diacritic, a classification of a part of an ideograph or other characteristic of a class of ideographs. 
Each of the sound indicators has a distinct hue. The operator may select any one of the displayed 
menu options, according to the Sound Match aspect of the invention, by speaking the sound 
5 associated with the sound indicator associated with the desired menu option. For example, assume 
the operator says, "blue" or a translation thereof, preferably a Chinese translation in this example. 
The ideograph located on the blue sound indicator is selected. In this example, this ideograph is 
input to a word processing program and appears on the display (21 12). 

10 The Sound Match aspect of the invention thus allows an operator to select from an option from a 
menu, using speech recognition means, whether or not the menu options are homophones. The 
operator does not need to use his hands to make this selection and so may keep both his hands on 
the home row of the keyboard, in preparation for entering the next common characteristic, or, if 
specifying the common characteristic by voice, may select a menu option without interrupting the 

15 manual activity he's engaged in. 



The preferred embodiment of the Sound Match aspect of the invention will now be described in 
detail from an implementation perspective, beginning with the hardware and software operating 
environment which will now be described with reference to Figure 62 which depicts a speech 

20 recognition system (6001) including a computer system (2116) and keyboard (2210), as earlier 
described, a sound board (6007) and a microphone (6009). The conventional computer system 
(2116) preferably includes an 80486 CPU running at 33MHz or faster, and is provided with Dragon 
Dictate Power Edition software, available from Dragon Systems, Inc., Newton, Massachusetts, 
USA. Preferably, the sound board is the Audio Capture Playback Adaptor and the microphone is 

25 the Shur Headset microphone, both available from Dragon Systems, Inc. Figure 63 depicts the 
software environment of the preferred embodiment, which includes the earlier described 
Windows® version 3.1 operating system (1204), an optional Windows® application program 
(6107), the Windows® Dragon software driver (6101) included in the Dragon Dictate Power 
Edition software, the Dragon Application (6103) included in the Dragon Dictate Power Edition 

30 software, and a speech recognition access program (6105). The Dragon Application (6103) is 
configured to match a sample of each of six sounds distinguishable by the Dragon Application 
(6103) which the operator can consistently produce. Using the example described above, these are 
the vowel sounds produced by the operator of e as it sounds in green, i as it sounds in while, u as ii 
sounds in blue, e as it sounds in red, o as it sounds in orange and e it sounds in grey. Each of these 
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sounds is associated respectively with an identifier, for example, a number or a sequence of one or 
more characters. The speech recognition access program (6105) is preferably a Windows® 
application program developed using, in part, the Voice Tools Software Development Kit available 
from Dragon Systems, Inc. At initialization time, the speech recognition access program (6105) 
5 defines an array of data structures defining the menu and submenu options and the menu hierarchy. 
For example, one of the elements of this array determines that, on selection, certain actions are to 
be taken, for example, inpuning text to an application program, and that certain submenu options 
and related data are to be associated with certain child edit windows. Also at initialization time, 
the speech recognition access program (6105) initializes callback procedures using the Dragon 
10 Application's Application Program Interface to receive notification from the Dragon Application 
when a sound has been matched. Also at initialization time, the main window of the speech 
recognition access program (6105) is preferably sized to just encompass the sound indicators 
shown in Figure 58. Also at initialization time, the speech recognition access program (6105) 
creates six child edit controls, each corresponding to one of the sound indicators shown in Figure 
15 58. Each of the child edit controls has the background color described above and a text color of 
black or white depending upon which provides better contrast against the background color of the. 
child edit control, and each is located on the display (2112) as shown in Figure 58. Also at 
initialization time, the menu options of the initial menu are each associated respectively with one of 
the child edit controls. After initialization, the speech recognition access program (6105), upon 
20 notification from the Dragon Application that a sound has been received and a match attempted, 

sequentially searches the list of identifiers matched to the sound by the Dragon Application (6103), 
starting with the best match, until it finds an identifier corresponding to any one of the six sounds 
distinguishable by the Dragon Application (6103). Assuming the operator says o as it sounds in 
orange, the Dragon Application (6103) provides to the speech recognition access program (6105) a 
25 list of matches including, before any other identifier corresponding to any one of the six sounds 

distinguishable by the Dragon Application (6103), the identifier associated with child edit control 
(5809). This child edit control is currently associated with menu option "vort x", the matched 
menu option. The speech recognition access program (6105) then sets the text of each of the child 
edit controls to one of the submenu options associated with the matched menu option. In the 
30 example above, submenu options space, "x", "r", "t", *'v" and "o" are associated with each of the 

child edit controls respectively and the text of the associated child edit control is set to the submenu 
option. Assuming the operator says e as in green, submenu option space is the matched submenu 
option and a space is input to the application program (6107) whose output appears in region 
(5850). Preferably the application program (6107) is an application program capable of executing 
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a WM_SIZE command so that the speech recognition access program (6105) may size the windows 
of the application program (6107) to fit neatly in region (5850) and is capable of executing 
WM_CHAR messages so that the speech recognition access program (6105) may input characters 
to the application program (6107). 

J. Ideographic Language 



The preferred embodiment of the Ideographic Language aspect of the invention will now be 
described in detail from a functional perspective using an example depicted in Figure 64. Figure 

10 64 shows the display (21 12) of a general purpose computer system (2218 in Figure 15) and 12 

selectable regions. Each of the 12 selectable regions consists of the union of a visible subregion on 
the display (2112) and an invisible subregion located outside the display (2112) and adjacent the 
visible subregion. For example, the selectable region in the upper left corner both above and below 
the top of the display (21 12) in Figure 64 consists of invisible subregion (3604) and visible 

15 subregion (3606), and hereinafter is referred to as selectable region (3604/3606). The other 
selectable regions shown in Figure 64, proceeding counter clockwise from selectable region 
(3604/3606) are (3608/3610), (3612/3614), (3616/3618), (3620/3622), (3624/3626), (3644/3646), 
(3648/3650), (3652/3654), (3656/3658), (3660/3662) and (3664/3666). In Figure 64 each of the 
visible subregions is adjacent an edge of the display (21 12). The selectable regions together 

20 circumscribe region (3680) in the center of the display. Also shown on the display (2112) within 
region (3680) in Figure 64 are ten square indicating regions arranged in two columns of five 
indicating regions. Each indicating region abuts an indicating region in the other column, the 
indicating region located above it. if any, and the indicating region located below it, if any. In the 
preferred embodiment, each indicating region indicates by its location the location of a respectiveK 

25 associated selectable region, in accord with the Location Indication aspect of the invention. For 
example, the uppermost indicating region in the left column of indicating regions (3605) is 
associated with the uppermost selectable region (3604/3606) on the left side of the display. 
Indicating region (3609) located immediately below indicating region (3605) is associated with 
selectable region (3608/3610) located immediately below selectable region (3604/3606). 

30 

The operation of the example of the preferred embodiment of the Ideographic Language aspect of 
the invention will now be described. Assuming that the operator is fitted with a head pointer 
coupled to the general purpose computer system (221 8). that he uses a keyboard to enter a phonetic 
unit and intonation according to the Pin Yin coding method for the Chinese language, and that he 
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enters the distinct sound "f6", each of the plurality of the selectable regions shown in Figure 64 is 
associated with one of a pluralit>' of menu options, each of the plurality of menu options having a 
common characteristic. In this example, the common characteristic is a phonetic unit and 
intonation but may be a phonetic unit alone, an intonation alone, a stroke used to draw an 
ideograph, a number of horizontal strokes, a number of vertical strokes, a number of total strokes, a 
stroke order, a radical, a part of speech, an ideograph, a kana. a diacritic, a classification of a part of 
an ideograph or other characteristic of a class of ideographs. The ten menu options shown in 
Figure 64 are Chinese ideographs each starting with the distinct sound "ftj*'. Alternatively, the 
menu options may be sequences of graphic symbols including one or more kanji. In Figure 64 the 
Chinese ideographs are each displayed on the visible subregion of the associated selectable region 
and on the indicating region associated with the selectable region. The remaining two of the 12 
selectable regions, (3624/3626) and (3644/3646), are associated with menu options for undo and 
for displaying more menu options, respectively. In the preferred embodiment, in response to the 
selection of the menu option for displaying more menu options, each of the selectable regions 
associated with a menu option starting with the distinct sound "f6*' is associated with a menu option 
not previously displayed and the newly associated menu option replaces the old menu option on the 
display. 

Resuming now with the description of the example of the preferred embodiment, a cross hair 
cursor (3686) is displayed in the circumscribed region (3680). Assuming the operator desires to 
select the menu option associated with selectable region (3608/3610), he turns his head to the left 
and the cross hair cursor (3686) moves to the left, responsive to the head movement, until the cross 
hair cursor hotspot intersects selectable region (3608/3610) and he maintains the location of the 
cross hair cursor hotspot on that selectable region for the selection threshold period. The menu 
option associated with selectable region (3608/3610) is selected and added to text (3684) displayed 
in the circumscribed region (3680), the general purpose computer system (2218) emits an audible 
beep indicating that selection has occurred and the displayed menu options, both on the indicating 
regions and the visible subregions, are removed from the display. 

In the preferred embodiment, selection is made in accord with the Facilitated Dwell subaspect of 
the Dwell aspect of the invention, described above. Thus, the operator receives an indication of the 
progress of his selection by a change in appearance of the indicating region associated with the 
intersected selectable region.. Alternatively, selection may be by intersection of a location 
indicated by the at least part of a cursor and a selectable region alone, by such an intersection 
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accompanied by a switch operation, for example, a depression of a space bar on the keyboard, or by 
other suitable means. 



In accord with the Location Indication aspect of the invention, the operator sees the entire menu in 

5 the compact indicating regions and may discover the location of the selectable region associated 
with each menu option without having to visually scan all the visible subregions. In the preferred 
embodiment, the plurality of indicating regions may be moved to a different location on the display 
to avoid obstructing the area of the display showing most recently added graphic symbols or the 
area of the display where graphic symbols will soon be added. In accord with the Ideographic 

10 Language aspect of the invention, an operator may select from among many sequences of one or 
more ideographs without lifting either hand from the keyboard, thus speeding entry of single 
ideographs or sequences of ideographs in word processing systems for the Chinese, Japanese and 
Korean languages. Since, in the preferred embodiment, the selectable regions are located adjacent 
the edge of the display, a large rectangular region remains available on the display for the output of 

15 an application program. Further, if the selectable regions are located entirely outside the display, 
the indicating regions obstruct only a relatively small portion of the circumscribed region (3680), 
permitting the display of a sequences of ideographs for selection simultaneous with the display of 
previously selected ideographs, neither obstructing the operator's view of the other. If the general 
purpose computer system is coupled to a speech synthesizer and the ideographs are symbols of a 

20 symbol set, for example, the Blissymbolics Symbol Set, an illiterate operator, for example, a school 
child having impaired speech, may select symbols associated with words and those words may be 
spoken via the speech synthesizer. 

Figure 65 illustrates a display and structures in accordance with an alternative embodiment of the 
25 Ideographic Language aspect of the invention. In this Figure, 1 8 selectable regions (3704/3706), 
(3708/3710), (3712/3714), (3716/3718), (3720/3722), (3724/3726), (3728/3730), (3732/3734), 
(3736/3738), (3740/3742), (3744/3746), (3748/3750),.(3752/3754), (3756/3758), (3760/3762), 
(3764/3766), (3768/3770), (3772/3774) and (3776/3778) circumscribe region (3780) on the display 
(21 12). The visible subregions of four of the 1 8 selectable regions abut each of the top, left and 
30 right edges of the display (21 12). The visible subregions of six of the 18 selecuble regions abut 
the bottom edge of the display (21 12). The 18 indicating regions are located within the top half of 
region (3780). 12 of the 18 indicating regions are arranged in two columns of six indicating 
regions each. The column on the left is close to the visible subregions abuning the left edge of the 
display. The column on the right is close to the visible subregions abutting the right edge of the 
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display. Two of the 18 indicating regions are located between the top indicating regions in each of 
the left and right columns. The remaining four of the 18 indicating regions are located between 
the bottom indicating regions in each of the left and right columns. Each of the four indicating 
regions in the top row of indicating regions indicates the location of a respectively associated 
5 selectable region along the top edge of the display. Each of the six indicating regions in the bottom 
row of indicating regions indicates the location of a respectively associated selectable region along 
the bottom edge of the display. Each of the middle four indicating regions in each of the left and 
right columns of indicating regions indicates the location of a respectively associated selectable 
region along the left and right edge of the display. The 16 menu options shown in Figure 65 are 
10 Chinese ideographs each of which includes the radical kou, a radical having the shape of a 

rectangle. Each ideograph is displayed on the visible subregion of the associated selectable region 
and on the indicating region associated with the selectable region. The remaining two of the 18 
selectable regions, (3724/3726) and (3744/3746), are associated with menu options for undo and 
for displaying more menu options, respectively. 



15 



20 



The indicating regions (3782) in Figure 65 are disproportionately large relative to the rest of Figure 
65. They are approximately 1 .5 times their proportionate size. They are represented as shown in 
Figure 65 in compliance with Patent Cooperation Treaty Rules requiring a minimum size for letters 
in figures. 



Figure 66 illustrates a display and structures in accordance with an alternative embodiment of the 
Ideographic Language aspect of the invention. Figure 66 depicts a display (21 12) of a computer 
system (2116) having thereon ten square indicating regions. (3801 ), (3803), (3805), (3807), (3809), 
(381 1), (3813), (3815), (3817), and (3819), arranged in two columns of five indicating regions. 

25 Each indicating region abuts an indicating region in the other column, the indicating region located 
above it, if any, and the indicating region located below it, if any. Each indicating region indicates 
the location of a respectively associated selectable region (not shown) outside the display. Assume 
that the operator is entering Chinese and uses a keyboard coupled to the computer system (2116) to 
enter the distinct sound "f6" . On each of the indicating regions is then displayed an ideograph 

30 having with the distinct sound "f6*' . The operator, using a pointer coupled to the computer system 
(21 16), points to the selectable region associated with the indicating region on which is displayed 
the desired ideograph. The selectable region is selected in response to a selection event, and, in 
this example, the associated menu option is input to a word processing program and appears on the 
display (2112). 
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Figure 67 illustrates a display and structures in accordance with an alternative embodiment of the 
Ideographic Language aspect of the invention. Figure 67 depicts a display (21 12) of a computer 
system (2116) having thereon eight selectable regions (6701), (6703), (6705), (6707), (6709), 
(671 1), (6713), and (6715), each located on the display (2112) adjacent the edge of the display 
5 (2112) and associated respectively with a menu option. Together the eight selectable region 

circumscribe region (6750) on the display. In Figure 67 each of eight menu options is displayed on 
its associated selectable region. Six of the eight menu options, associated with selectable regions 
(6703). (6705), (6707), (6701), (6715). and (6713) each represent a sequence of graphic symbols in 
the Japanese language. These sequences, in Romanized Japanese are respectively "hoka". "hoka 

10 ni", "hoka no", "hoka kara", "nanika hoka nomono", and "hoka demo nai ga". Each sequence 

includes the kanji "hoka" shown alone on selectable region (6703), The sequences are ordered by 
length, the shorter sequences on selectable regions on the left side of the display (2112) ordered by 
length from the top to the bottom of the display (21 12). the longer sequences along the right side of 
the display (2112) ordered by length from the lop to the bottom of the display (2112). The 

15 remaining two of the eight selectable regions, (6709) and (671 1 ), are associated with menu options 
for displaying the previous and the next display of menu options, respectively. The menu option 
associated with a selectable region may be selected by a selection event. 

The preferred embodiment of the Ideographic Language aspect of the invention will now be 
20 described in detail from an implementation perspective. Preferably, the Ideographic Language 

aspect of the invention is implemented by modifications to the access program (1206) described in 
the detailed description of the Perimeter Menu aspect of the invention, modified as described in the 
descriptions of the Facilitated Dwell subaspect and the Location Indication aspects of the 
invention. The modifications required for the Ideographic Language aspect of the invention are: ( 1 ) 
25 install on the general purpose computer system a font for the ideographic language of the 

embodiment; (2) at initialization time: (a) set the text of all labels for display on the selectable 
regions to null; (b) do not enable the cursor polling timer; and (c) hide the Windows® ( 1 204) 
system cursor; (3) include in the main window processing procedure of the access program (1206) 
code to process WM_CHAR messages and, when a sequence of one or more WM_CHAR 
30 messages indicates a common characteristic: (a) lookup sequences or representations of sequences 
of one or more ideographs (hereinafter "sequences") having the common characteristic: (b) cop\ 
ten of the sequences to the labels; (c) set the Windows® (1204) system cursor to the cross hair 
cursor, set the cursor location to a predetermined location near an indicating region and show the 
Windows® ( 1 204) system cursor: (d) set fPaini to TRUE for every state machine: (e) send 
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EV RESET to every state machine; (0 display each sequence at the appropriate location in the 
indicating region: and (g) enable the cursor polling timer; and (4) add state processing to 
ST_SELECTED to: (a) set the text of all labels for display to null; (b) hide the Windows® (1204) 
system cursor; (c) erase all sequences from the indicating region; (d) disable the cursor polling 
5 timer; (e) set fPaint to TRUE for every state machine; (0 send EV RESET to every state machine: 
and (g) insert the selected sequence into the work space. 

In the above descriptions, there is shown and described only the preferred and certain alternate 
embodiments of each aspect of the invention, but. as aforementioned, it is to be understood that 
10 each aspect of the invention is capable of use in various other combinations and environments and 
is capable of changes or modifications within the scope of the inventive concepts as expressed 
herein. 
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Appendix I 



Procedure : PocketFsm 



20 



/* 

/* Procedure: PocketFsm 



PocketFsm (pPocket , Event) 

{ 



*/ 

*/ 



/♦ Descripcion: State Machine for processing events to */ 

10 /♦ selectable regions. */ 

/* V 

/♦ Input Parameters: */ 

/* pPocket pointer to selectable region data */ 

/* set */ 

15 /* Event event to process */ 

*/ 



BOOL 



f InternalEvent 



/* declaration of local 
boolean variable */ 



set f InternalEvent to TRUE 



25 



30 



while ( f InternalEvent is TRUE) 

{ 

set f InternalEvent to FALSE 

set pPocket - >PreviousState to pPocket - >State 

set pPocket- >State to aPocketFsm [pPocket - >State] (Event ] 

switch (pPocket - >State) 



{ 



case ST_ILLEGAL_STATE: 
case ST ERROR: 
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turn off all timers 
log the error 

display an error message to the operator 
break 

case ST_INITIAL;: 
break 

case ST_RESET: 

set time of selection to the current time 
if pPocket- >Color is not equal to 
pPocket- >InitialColor 

set pPocket ->Color to pPocket- >InitialColor 

sec pPocket- >f Paint to TRUE 
set pPocket- >f Invert to FALSE 
break 

case ST_LOW_TIDE: 
break 

case ST_CREST_TIDE : 
break 

case ST_BEGIN_LOCK: 

emit an audible beep 

set the system cursor location to the visible 

subregion's lockspot 

set the system cursor to null 

display the lock icon on the visible subregion's 
lockspot 



/* initialize counter 



for locking ♦/ 



set iLockCursor to cLockCursor 



break 
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case ST_LOCK: 

set the system cursor location to the visible 
subregion ' s lockspot 

/* on lock coxanter 
5 expiration, */ 

/* transfer to the next 
state */ 
decrement iLockCursor by 1 
if iLockCursor equals 0 
10 set Event to EV_NULL 

set f InternalEvent to TRUE 
break 



case ST_END_LOCK: 
15 erase the lock icon with white 

set the system cursor to the arrow cursor 
display the system cursor on the visible 
subregion ' lockspot 
emit an audible beep 
20 /* paint selectable region 

so that ♦/ 

/♦ the white erasure of the 
lock */ 

/* cursor icon is 
25 overwritten */ 

set pPocket - >f Paint to TRUE 
break 



case ST_SELECTED: 
30 emit an audible beep 

take the action appropriate upon selection of this 
selectable region, for example, sending data to an 
application program, sending a control sequence to 
a device coupled to the computer, or displaying 
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the selection 

sec pPocket- >f Invert to TRUE 

if appropriate, change the menu options associated 

with various selectable regions and set fPaint to 

TRUE for the state machines associated with those 

selectable regions 

set Event to EV_rJULL 

set f InternalEvent to TRUE 

break 

case ST_SELECT_AND_OUT : 

send EV_RESET then EV_MOVEMENT to all state 
machines, resetting them and indicating that the 
operator is moving the cursor 
break 

case ST_DECAY: 

/* set state to previous 
state */ 

/* in preparation for the 
next ♦/ 

/* state transition ♦/ 
set pPocket - >State to pPocket- >PreviousState 
decrement pPocket - >Color by pPocket - ^Decrement , 
but not below pPocket - >InitialColor 
if pPocket- >Color was changed 

set pPocket- >f Paint to TRUE 

set f InternalEvent to TRUE 
if pPocket - >Color was changed from a value greater 
than or equal to pPocket- >CrestTide to a value 
below pPocket - >CrestTide 

set Event to EV_STEP_DOWN 

else 

set Event to EV NULL 
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break 

case ST_DWELL: 

/* set state to previous 
5 state */ 

/* in preparation for the 
next */ 

/* state transition ♦/ 
set pPocket - >State to pPocket - >PreviousState 
10 increment pPocket - >Color by pPocket- >Increment , 

but not above pPocket - >Ceiling 
if pPocket- >Color was changed 

set pPocket - >f Paint to TRUE 
set f InternalEvent to TRUE 
15 if pPocket - >Color was changed from a value below 

pPocket- >CrestTide to a value greater than or 
equal to pPocket - >CrestTide 
set Event to EV_STEP__UP 

else 

20 set Event to EV_NULL 

break 

case ST_IDLE: 

set the system cursor location to the center of 
25 the display 

set f InternalEvent to TRUE 
set Event to EV_NULL 
break 

30 case ST_EBB_TIDE: 

break 

case ST_ENTRY: 

set f InternalEvent to TRUE 
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sec Event to EV NULL 



break 



case ST DISCARD: 



/* set state to previous 



state */ 



/* for the next transition*/ 



to 



15 



20 



set pPocket - >State to pPocke t - >PreviousState 
break 

case ST_EXIT: 

set Event to EV_NULL 

set f InternalEvent to TRUE 

if pPocket- >f Interior equals TRUE 

change the boundaries of the Windows® region 
having the handle pPocket- >hRegion to chose 
defined by the second array of points 
associated wich Chis staCe machine 

sec pPockec- >f Paine CO TRUE 

send EV_RESET co Che scace machine having Che, 

index pPockec- >iAdj acenCPockeC 

break 

def aulc : 

display error message 
break 



} 



/* end swicch ♦/ 



/* end while */ 



if pPocket - >f Paint is equal to TRUE 



invalidate the entire client area 
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/♦ end PocketFsm */ 
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Procedure : CreateEvent 



/* */ 

5 /* Procedure: CreateEvent ♦/ 

/* V 

/♦ Description: Examines movement of cursor and determines ♦/ 

/* what events have occurred. ♦/ 

/* */ 

10 /♦ Input Parameters: ♦/ 

/* pEvent pointer to an event data structure */ 

/* for output ♦/ 

/♦ Output: A completed event data structure */ 

/* indicated by pEvent including: ♦/ 

15 /♦ (1) an indication of whether the cursor hotspot */ 

/* intersects a selectable region . ^ _ - - ♦/ 

/* (2) the index of the intersected selectable ♦/ 

/* region, if any */ 

/* (3) the event for the intersected selectable ♦/ 

20 /♦ region, if any */ 

/* (4) the event for all non- intersected */ 

/* selectable regions */ 

/* V 

/* Updated variables indicate: */ 

25 /* (1) the previous location of the cursor hotspot */ 

/* (2) the approximate time of the sampling of the */ 

/* previous location of the cursor hotspot */ 

/* */ 

30 CreateEvent (pEvent) 

{ 

get current cursor hotspot location 

if there has been cursor hotspot movement since the last 
cursor location 
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sampl ing 

set time of previous cursor movement to time latest 
received 

WM__TIMER message was generated 
5 if cursor hotspot has crossed out of a selectable region 

set event to EV_CROSS_OUT for the intersected 
selectable region and to EV_MOVEMENT for all other 
selectable regions 

else 

10 if cursor hotspot intersects a selectable region 

if intersected selectable region's Color equals 
Ceiling 

set event to EV_CEILING for the intersected 
selectable region and to EV_DECAY for all 
15 other selectable regions 

else 

set event to EV_DWELL for the intersected 
selectable region aind to EV^DECAY for all 
other selectable regions 

20 else 

set event to EV_MOVEMENT for all selectable 
regions 

else 

if cursor hotspot intersects a selectable region 
25 if idle time exceeded 

set event to EV__IDLE_TIMEOUT for all selectable 
regions 

else 

if intersected selectable region's Color equals 
30 Ceiling 

set event to EV_CEILING for the intersected 
selectable region and to EV_DECAY for all 
other selectable regions 

else 



no 
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set event to EV^DWELL for the intersected 
selectable region and to EV_DECAY for all 
other selectable regions 

else 

5 set event to EV_DECAY for all selectable regions 

set the previous cursor hotspot location to the current 
hotspot location 

} 

10 /* end CreateEvent */ 
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Claims 



I claim: 



10 
15 

2. 

20 

25 3. 
4. 

30 



For use with a human interface system wherein a body member of an operator may indicate a 
first location on a surface, the surface including a display area, a method of selecting a 
sequence of one or more graphic symbols from a plurality of sequences of one or more 
graphic symbols, one or more sequences of the plurality of sequences including one or more 
ideographs, said method comprising ihe steps of: 

displaying on the display area a plurality of selectable regions, the plurality of selectable 
regions together at least partially circumscribing a region on the display area, at least two of 
the selectable regions associated respectively with one sequence of the plurality of 
sequences, each of the plurality of sequences having a first common characteristic; and 

in response to a selection event wherein the first location indicated by the body member at or 
near the time the selection event occurs intersects any one of the at least two selectable 
regions, selecting the sequence associated with the intersected selectable region. 

The method of claim 1 wherein each of the sequences belongs to any one of the Chinese 
language, Japanese language, Korean language. Picture Communication Symbols symbol set. 
Rebus symbol set, Picsyms symbol set, Pictogram Ideogram Communication Symbols 
symbol set, Ycrkish symbol set, Blissymbolics symbol set. and a set of depictions of the 
signs of a manual sign language. 

The method of claim 1 further comprising confming the first location indicated by the body 
member of the operator to the display area. 

The method of claim I wherein the operator keeps at least five of his fingers, not including 
either thumb, on the home row of a keyboard while moving the body member to indicate the 
first location on the surface. 

The method of claim 1 wherein the first common characteristic includes any one of: 
(a) a phonetic unit, including either one of a Chinese pronunciation and a Japanese 
pronunciation; 
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(b) 


an intonation: 


(c) 


a stroke used to draw an ideograph; 


(d) 


a number of horizontal strokes;. 


(e) 


a number of vertical strokes; 


(0 


a number of total strokes; 


(g) 


a stroke order; 


(h) 


a radical; 


(i) 


a pan of speech; 


0) 


an ideograph; 


(k) 


a kana; 


(1) 


a diacritic; 


(m) 


a classification of a part of an ideograph; 


(n) 


a meaning; and 


(o) 


a meaning class. 



The method of claim 1 wherein the body member of the operator is a body member other 
than the shoulder, arm, elbow, wrist, hand, finger, or thumb. 

The method of claim 6 wherein the body member of the operator is the head of the operator. 

The method of claim I wherein the displaying step further comprises displaying a first cursor 
on the display area at the first location indicated by the body member; and wherein the 
selecting step further comprises removing the first cursor from the display area. 

The method of claim 8 wherein the selecting step further comprises displaying a second 
cursor at a predetermined location on the display area. 

The method of claim 1 further comprising displaying on the display area one sequence of the 
plurality of sequences associated with one of the at least two selectable regions. 

The method of claim 10 wherein the selecting step further comprises displaying a first cursor 
on the display area at a location on or near the displayed sequence. 

The method of claim 10 wherein the step of displaying one sequence of the plurality of 
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sequences further comprises indicating the association of the displayed sequence with its 
associated selectable region. 

The method of claim 12 wherein the indicating step further comprises displaying any part of 
either of: 

(a) the foreground of the displayed sequence; and 

(b) the background of the displayed sequence 
in any one of: 



(I) 


a location: 


(2) 


a size; 


(3) 


a shape; 


(4) 


a hue; 


(5) 


a brightness: 


(6) 


a contrast; 


(7) 


a tone; 


(8) 


a dithering; 


(9) 


a fill; and 


(10) 


a font 



associated with the associated selectable region. 

The method of claim 1 further comprising selecting the first common characteristic from a 
plurality of common characteristics of ideographs. 

The method of claim 1 wherein the selection event includes a switch operation. 

The method of claim 1 wherein at least one sequence of the plurality of sequences includes 
one or more graphic symbols in addition to the graphic symbols of the at least one sequence 
representing the first common characteristic. 

The method of claim 16 wherein the first common characteristic includes an ideograph and 
the additional graphic symbols of the at least one sequence include any one of: 
(a) a prefix; 
(b an infix: and 
(c) a suffix: 
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which may be added to the ideograph. 

18. The method of claim 1 wherein the at least two of the selectable regions are ordered 
responsive to any one of: 

(a) the frequency of use of the associated sequence in an ideographic language; 

(b) the frequency of use of the associated sequence in a predetermined operator's usage: 

(c) the recency of use of the associated sequence in a predetermined operator's usage: 

(d) the length of associated sequence; 

(e) the alphabetic order of the associated sequence: and 
(0 a second common characteristic. 

19. For use with a human interface system including a keyboard wherein a body member of an 
operator may indicate a first location on a surface, the surface including a display area, a 
method of selecting a sequence of one or more graphic symbols from a pluraiit>' of sequences 
of one or more graphic symbols, one or more sequences of the plurality of sequences 
including one or more ideographs, said method comprising the steps of: 

displaying on the display area a plurality of selectable regions, at least two of the selectable 
regions associated respectively with one sequence of the plurality of sequences, each of the 
plurality of sequences having a common characteristic; 

moving the body member to indicate the first location on the surface while at least five of the 
operator's fingers, not including either thumb, remain on the home row of the keyboard: and 

in response to a selection event wherein the first location indicated by the body member at or 
near the time the selection event occurs intersects any one of the at least two selectable 
regions, selecting the sequence associated with the intersected selectable region. 

20. The method of claim 1 or 19 wherein the body member may indicate a second location on the 
surface and wherein the selection event includes the duration of a period of intersection 
equalling or exceeding a selection threshold period, the period of intersection starting in 
response to the first location intersecting any one of the selectable regions and ending in 
response to the second location intersecting the selectable region intersected by the first 
location. 
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21. For use with a human interface system wherein a body member of an operator may indicate a 
location outside a display area, a method of selecting a sequence of one or more graphic 
symbols from a plurality of sequences of one or more graphic symbols, one or more 
5 sequences of the plurality of sequences including one or more ideographs, said method 

comprising the steps of: 



at least partially delimiting a pluralit> of selectable regions outside the display area, at least 
two of the selectable regions associated respectively with one sequence of the plurality of 
10 sequences, each of the plurality of sequences having a common characteristic, and 

in response to a selection event wherein the location indicated by the body member at or near 
the time the selection event occurs intersects any one of the at least tv^'o selectable regions, 
selecting the sequence associated with the intersected selectable region. 



15 



The method of claim 21 wherein the body member of the operator is a body member other 
than either of the operator's eyes. 



23. For use with a human interface system wherein a body member of an operator may indicate 
20 successive locations on a surface, the surface including a display area, a method of selecting 

a sequence of one or more graphic symbols from a plurality of sequences of one or more 
graphic symbols, one or more sequences of the plurality of sequences including one or more 
ideographs, said method comprising the steps of: 

25 at least partially delimiting a plurality of selectable regions, each of the selectable regions 

including an invisible subregion adjacent the display area and a visible subregion on the 
display area, the plurality of visible subregions together at least partially circumscribing a 
region on the display area, at least two of the selectable regions associated respectively with 
one sequence of the plurality of sequences, each of the plurality of sequences having a 

30 common characteristic: and 



responsive to the duration of a period of intersection equalling or exceeding a selection 
threshold period, the period of intersection starting in response to a starting location 
intersecting any one of the invisible subregions included in one of the at least two selectable 
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regions and ending in response to an ending location intersecting the invisible subregion 
intersected by the starting location, the starting and the ending locations each being one of 
the successive locations, selecting the sequence associated with the selectable region 
including the invisible subregion intersected by the starting location. 

24. The method of claim 23 further comprising indicating the duration of a period of 
intersection, said period staning in response to the starting location and ending in response to 
an intermediate location intersecting the invisible subregion intersected by the starting 
location, the intermediate location being one of the successive locations occurring after the 
staning location. 

25. For use with a human interface system wherein a body member of an operator may indicate 
successive locations on a surface, the surface including a display area, a method of selecting 
a sequence of one or more graphic symbols from a plurality of sequences of one or more 
graphic symbols, one or more sequences of the plurality of sequences including one or more 
ideographs, said method comprising the steps of: 

displaying on the display area a plurality of selectable regions, the plurality of selectable 
regions together at least partially circumscribing a region on the display area, at least two of 
the selectable regions associated respectively with one sequence of the plurality of 
sequences, each of the plurality of sequences having a common characteristic; and 

responsive to a quantity equalling or exceeding a selection threshold period, said quantity 
being a function of the durations of a first and a second period of intersection 



said first period of intersection starling in response to a first starting location 
intersecting any one of the at least two selectable regions and ending in response to a 
first ending location intersecting the selectable region intersected by the first starting 
location, 

said second period of intersection starting in response to a second starting location 
intersecting the selectable region intersected by the first starting location and ending in 
response to a second ending location intersecting the selectable region intersected b\ 
the first starting location. 
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the first starting location, the second starting location, the first ending location and the 
second ending location each comprising one of the successive locations, 

selecting the sequence associated with the selectable region intersected by the first starting 
5 location. 

26. The method of claim 25 further comprising increasing the selection threshold period 
responsive to the duration of an intermediate period of non-intersection of the selectable 
region intersected by the first starting location, said intermediate period of intersection 

10 starting in response to an intermediate starting location not intersecting the selectable region 

intersected by the first staning location and ending in response to an intermediate ending 
location not intersecting the selectable region intersected by the first starting location, the 
intermediate starting and the intermediate ending locations each comprising one of the 
successive locations and each occurring at the time or after the first ending location occurs 

15 and before or at the time the second staning location occurs. 

27. The method of claim 26 further comprising indicating the duration of the intermediate period 
of non- intersection, 

20 

28. A method of selecting a sequence of one or more graphic symbols from a plurality of 
sequences of one or more graphic symbols, one or more sequences of the plurality of 
sequences including one or more ideographs, said method comprising the steps of 

25 displaying on a display a plurality of selectable regions, each of the selectable regions 

associated respectively with one sequence of the plurality of sequences, each of the plurality 
of sequences having a common characteristic; 

receiving a movement related signal and moving a cursor on the display responsive thereto; 
30 and 

selecting the sequence associated with the particular one of the plurality of selectable regions 
most nearly along a path indicated by a cursor location and the movement related signal in 
advance of an intersection of the cursor and said particular selectable region. 
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A method of selecting a sequence of one or more graphic symbols from a pluralit>' of 
sequences of one or more graphic symbols, one or more sequences of the plurality of 
sequences including one or more ideographs, said method comprising the steps of: 

displaying on a display a plurality of selectable regions, each of the selectable regions 
associated respectively with one sequence of the plurality of sequences, each of the pluralitv 
of sequences having a common characteristic: 

receiving a movement related signal and moving a cursor on the display responsive thereto: 
and 

responsive to: 

(a) an intersection of the cursor and a selectable region; and 

(b) a cursor path toward the intersected selectable region, 

selecting the sequence associated with the intersected selectable region. 

The method of claim 29 wherein the selecting step is further responsive to the duration of the 
period of the intersection equalling or exceeding a selection threshold period. 

For use with a human interface system wherein a body member of an operator may indicate 
successive locations on a surface, the surface including a display area, a method of selecting 
an undo function, said method comprising the steps of: 

first displaying on the display area a plurality of selectable regions, the plurality of selectable 
regions together at least partially circumscribing a region on the display area, at least two of 
the selectable regions associated respectively with one sequence of a plurality of sequences 
of graphic symbols, one or more sequences of the plurality of sequences including one or 
more ideographs and each having a common characteristic; 

in response to a first selection event wherein a first one of the successive locations indicated 
by the body member at or near the time the first selection event occurs intersects any one of 
the at least two selectable regions, selecting the sequence associated with the intersected 
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selectable region: 

performing a function with the selected sequence: 

second displaying on the display area a selectable region associated with the undo function: 
and 

in response to a second selection event wherein a second one of the successive locations 
indicated by the body member at or near the lime the second selection event occurs intersects 
the selectable region associated with the undo function, undoing the function performed with 
the selected sequence. 

2. A method of data entry in an ideographic language, said method comprising the steps of: 

displaying on a display a plurality' of selectable regions, the plurality of selectable regions 
together at least partially circumscribing a region on the display, at least two of the selectable 
regions associated respectively with one sequence of a plurality of sequences of graphic 
symbols, one or more sequences of the plurality of sequences including one or more 
ideographs and each having a common characteristic; 

receiving a movement related signal and moving a cursor on the display responsive thereto: 

responsive to a cursor path toward any one of the at least two selectable regions, changing a 
selection threshold period associated or to be associated with said selectable region: and 

responsive to the duration of a period of intersection of the cursor and said selectable region 
equalling or exceeding the selection threshold period, entering into the at least partially 
circumscribed region the sequence associated with said selectable region. 

3. For use with a speech synthesis system including a speech synthesizer and a display, a 
method of speaking comprising the steps of: 

displaying on the display a plurality of selectable regions, the plurality of selectable regions 
together at least partially circumscribing a region on the display, at least two of the selectable 
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regions associated respectively with one sequence of a plurality of sequences of graphic 
symbols, one or more sequences of the plurality of sequences including one or more 
ideographs and each having a common characteristic; 

moving a cursor on the display responsive to the movement of a body member of an 
operator; and 

in response to a selection event wherein the cursor at or near the time the selection event 
occurs intersects any one of the at least two selectable regions, speaking, by means of the 
speech synthesizer, a word associated with the sequence associated with the intersected 
selectable region. 

An apparatus for selecting a menu option from a plurality of menu options, said apparatus 
comprising: 

(a) display means for displaying on a display the plurality of menu options, each of the 
plurality of menu options associated respectively with a sound; 

(b) sound receiving means for receiving a sound signal; 

(c) match means for matching the received sound signal to any one of the plurality of 
sounds; and 

(d) selection means for selecting the menu option associated with the matched sound. 

The apparatus of claim 34 wherein one or more of the menu options represents a sequence of 
one or more letters and the sound associated w ith each of the one or more menu options is 
not a phonetic representation of the menu option. 

The apparatus of claim 34 wherein each of the plurality of sounds is associated respectively 
with a sound indicator other than the displayed menu option. 

The apparatus of claim 36 further comprising indication means for indicating the association 
of each sound indicator to the menu option associated with the sound associated w ith the 
sound indicator. 

The apparatus of claim 37 w herein each of the sound indicators is outside the display. 
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39. The apparatus of claim 36 wherein one or more of the sound indicators includes a non- 
alphanumeric indicator. 

40. The apparatus of claim 39 wherein the non-alphanumeric indicator includes any one of a hue 
indicator, a pitch indicator, a volume indicator, a sound duration indicator, a change in pitch 
indicator, a change in volume indicator, a shape indicator, and an object indicator. 

41. For use with a general purpose computer system including a displa\. an apparatus for 
selecting a menu option from a plurality of menu options, said apparatus comprising: 

(a) display means for displaying on the display the plurality of menu options, each of the 
plurality of menu options associated respectively with a hue displayed intersecting or 
near the displayed menu option; 

(b) sound receiving means for receiving a signal representing a sound; 

(c) match means for matching the received sound to a name of any one of the pluralirv of 
hues; and 

(d) selection means for selecting the menu option associated with the hue corresponding to 
the matched name. 

42- The apparatus of claim 34 or 41 wherein the plurality of displayed menu options together at 
least partially circumscribe a region on the display. 

43. An apparatus for selecting a menu option from a plurality of menu options, said apparatus 
comprising: 

(a) a surface on which may be displayed the plurality of menu options, each of the 
plurality of menu options associated respectively with a sound indicator other than the 
displayed menu option; 

(b) a sound matcher for matching a sound to any one of the plurality of sound indicators; 
and 

(c) a selector for selecting the menu option associated with the matched sound indicator. 

44. For use with a computer capable of executing an application program, a computer access 
device, said device comprising: 

(a) a menu hierarchy including a menu including a pluralir>' of menu options, one or more 
of the menu options associated respectively with a sound which is not a phonetic 
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representation of the menu option and with a submenu including a plurality of 
submenu options, each of the submenu options associated respectively with a sound 
which is not a phonetic representation of the submenu option and with a sequence of 
one or more characters: 
5 (b) sound receiving means for receiving a first and a second sound signal; 

(c) match means for matching the first sound signal to any one of the sounds associated 
with the one or more menu options and for matching the second sound signal to any 
one of the sounds associated with the submenu options; and 

(d) input means, responsive to the match means matching the first and the second sound 
10 signals, for inputting to the application program the sequence of one or more 

characters associated with the submenu option associated with the sound matched by 
the second sound signal. 

45. The computer access device of claim 44 further comprising: 
15 (e) a display: and 

(0 means for displaying the one or more menu options on the display, and, responsive to 
the match means matching the first sound signal, displaying the submenu options on 
the display, 

20 46. For use with a general purpose computer system including a display on which may be 

displayed a plurality of menu options; and including sound receiving means for receiving a 
sound signal, an apparatus for selecting a menu option from the plurality of menu options, 
said apparatus comprising: 

(a) a medium readable by the general purpose computer system; and 
25 (b) a program, stored on the medium and executable by the general purpose computer 

system, for: 

(I) displaying on the display the plurality of menu options, each of the plurality of 
menu options associated respectively with a sound which is not a phonetic 
representation of the menu option; 
30 (2) matching the received sound signal to any one of the plurality of sounds; and 

(3) selecting the menu option associated with the matched sound. 

47. The apparatus of claim 46 wherein the medium includes a random access memorv . 
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48. The apparatus of claim 46 wherein the medium is coupled to the general purpose computer 
system. 

49. The apparatus of claim 46 wherein the medium may be coupled to and uncoupled from the 
5 general purpose computer system. 

50. The apparatus of claim 46 wherein the medium includes either one of a magnetic store and 
an optical store. 

10 51. A speech synthesis system comprising: 

(a) a display for displaying a plurality of menu options, each of the menu options 
associated respectively with a sound and with a sequence of one or more characters; 

(b) sound receiving means for receiving successive sound signals; 

(c ) a speech synthesizer; and 
15 (d) control means for 

( 1 ) matching one of the successive sound signals to any one of the plurality of 
sounds; 

(2) repetitively 

(I) selecting the sequence of one or more characters associated with the menu 
20 option associated with the matched sound; and 

(ii) appending the selected sequence to at least one previously selected 
sequence; and 

(3) speaking, by means of the speech synthesizer, the word spelled by the appended 
sequences. 

25 

52. A method of selecting a sequence of one or more graphic symbols from a plurality of 
sequences of one or more graphic symbols, one or more sequences of the pluralit\' of 
sequences including one or more ideographs, said method comprising the steps of: 

30 displaying on a display the pluralit>' of sequences, each of the plurality of sequences having a 

common characteristic and each associated respectively with a sound indicator; 

matching a sound to any one of the pluralit>' of sound indicators: and 
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selecting the sequence of the plurality of sequences associated with the matched sound 
indicator. 



53. The method of claim 52 wherein the sound is not a phonetic representation of the sequence 
associated with the matched sound indicator. 

54. The method of claim 52 wherein the sound indicator is non-alphanumeric. 

55. The method of claim 54 wherein the sound indicator is a hue. 

56. The method of claim 52 further comprising the step of indicating the association of each of 
the plurality of sequences with the associated sound indicator. 

57. The method of claim 52 w herein each of the sequences belongs to any one of the Chinese 
language, Japanese language, Korean language. Picture Communication Symbols symbol set. 
Rebus symbol set, Picsyms symbol set, Pictogram Ideogram Communication Symbols 
symbol set, Yerklsh symbol set, Blissymbolics symbol set, and a set of depictions of the 
signs of a manual sign language. 

58. An apparatus for selecting a menu option from a plurality of menu options, said apparatus 
comprising: 

(a) a display area; 

(b) means for at least partially delimiting a plurality of selectable regions, each of the 
selectable regions outside the display area and each associated respectively with a 
menu option; 

(c) movement related signal receiving means for receiving a movement related signal 
indicating a location with respect to the display area; and 

(d) selection means for selecting, in response, to a selection event associated with the 
selectable region intersected by the location, the menu option associated with the 
intersected selectable region. 



59. The apparatus of claim 58 wherein the selection event includes a switch operation at or near 
the time the intersection occurs. 
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60. The apparats of claim 58 further comprising means for indicating the menu option associated 
with each selectable region, 

61 . The apparatus of claim 58 further comprising means for indicating which one of the 
5 selectable regions is intersected by the location. 

62. The apparatus of claim 58 further comprising location indication means for indicating the 
location of each selectable region. 

10 63. The apparatus of claim 62 wherein the location indication means further comprises means for 
displaying each menu option on the display area, wherein the location of each displayed 
menu option indicates the location of the associated selectable region, 

64 The apparatus of claim 62 where the location indication means includes means for displaying 
1 5 at least part of each selectable region on the display area. 

65. The apparatus of claim 58 wherein the selection event includes a switch operation: and 
wherein the selection means further comprises switch operation receiving means for 
receiving a signal indicating the switch operation. 

20 

66. The apparatus of claim 58 wherein the selection means is at least partially disabled in 
response to a first selection event. 

67. The apparatus of claim 66 wherein the selection means, in response to a second selection 
25 event, is restored to the functionality it had prior to the first selection event. 

68. The apparatus of claim 58 wherein all or all but one of the selectable regions are partially 
delimited. 

30 69. The apparatus of claim 68 wherein one of the selectable regions is completely delimited. 

70. The apparatus of claim 68 further comprising a computer system including display means for 
displaying at least part of the output of an application program executable on the computer 
system in the region on the display area and wherein at least one of the menu options 
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represents an input to the application program. 

71 . An apparatus for selecting a menu option from a plurality of menu options, said apparatus 
comprising: 
5 (a) a display area; 

(b) means for at least partially delimiting a plurality of selectable regions, each of the 
selectable regions outside the display area and each associated respectively with a 
menu option: 

(c) movement related signal receiving means for receiving a movement related signal 
10 indicating successive locations with respect to the display area; and 

(d) selection means, responsive to a first dwell event associated with any one of the 
selectable regions outside the display area, for selecting the menu option associated 
with the selectable region outside the display area associated with the first dwell event. 

15 72- The apparatus of claim 71 further comprising a pointer, responsive to the movement of a 
body member of an operator other than either of the operator's eyes,. for generating the 
movement related signal. 

73. The apparatus of claim 71 wherein the selection means is responsive to a plurality of periods 
20 of intersection, each of two or more of the successive locations and the intersected selectable 

regions- 

74. The apparatus of claim 71 wherein some of the selectable regions are not completely visible. 

25 75. The apparatus of claim 71 wherein at most one of the selectable regions is adjacent the 
display area. 

76. The apparatus of claim 71 wherein each of the successive locations is relative to a 
predetermined location on the display area or to a previous location of the successive 

30 locations. 

77. The apparatus of claim 71 wherein the first dwell event includes a first quantirv' equalling or 
exceeding a predetermined quantit\ , the first quantity being a function of the durations of 
one or more successive periods of intersection of two or more of the successive locations and 
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one of the selectable regions: and wherein the selectable region associated with the first 
dwell event is the intersected selectable region. 

78. The apparatus of claim 71 wherein the first dwell event includes a first quantit\ equalling or 
5 exceeding a predetermined quantit\-, the first quantity being a function of: 

( 1 ) the durations of one or more successive periods of intersection of two or more of the 
successive locations and a first one of the selectable regions: and 

(2) the durations of one or more successive periods of intersection of two or more of the 
successive locations and a second one of the selectable regions: and 

10 wherein the selectable region associated with the first dwell event is the selectable region 

intersected by one of the successive locations when the first dwell event occurs. 

79. The apparatus of claim 71 further comprising a pluralit\' of selectable regions on the displa> 
area , each associated respectively with one of the selectable regions outside the display area: 

15 and wherein the selection means is further operative, responsive to a second dwell event 

associated with any one of the selectable regions on the display area, to select the menu 
option associated with the selectable region outside the display area associated with the 
selectable region on the display area associated with the second dwell event. 

The apparatus of claim 79 wherein the selection means is further operative, responsive to a 
first quantity equalling or exceeding a predetermined quantity, the first quantity being a 
function of: 

(a) the durations of one or more successive periods of intersection of two or more of the 
successive locations and one of the selectable regions on the display area: and 

(b) the durations of one or more successive periods of intersection of two or more of the 
successive locations and the selectable region outside the display area associated with 
the intersected selectable region on the display area; 

to select the menu option associated with the selectable region outside the display area. 

30 81. The apparatus of claim 79 wherein each of one or more of the selectable regions on the 
display area is adjacent the associated selectable region outside the display area. 

82. The apparatus of claim 79 wherein each of one or more of the selectable regions on the 

display area indicates the location of the associated selectable region outside the display area. 



20 80. 



25 
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83 The apparatus of claim 79 wherein the plurality of selectable regions on the display area 
together at least partially circumscribe a region on the display area. 

84. The apparatus of claim 83 wherein the selection means further includes means for indicating 
the remaining dwell time required to select the intersected selectable region. 

85. The apparatus of claim 83 w herein the movement related signal is responsive to the 
movement of a body member of an operator having impaired abilitx' to sense the position of 
the body member and the apparatus further comprises tactile indication means for indicating 
tactilely to the operator the position of the body member. 

86. The apparatus of claim 83 wherein the selection means further includes means for indicating 
on the display area the location of one of the successive locations located outside the display 
area. 



87 The apparatus of claim 83 wherein the selection means further includes means for indicating 
on the display area the distance between one of the successive locations located outside the 
display area and the point on the display closest thereto. 

88. In a human interface system including a display whereon a first cursor may be displayed and 
moved responsive to successive locations indicated by a movement related signal an 
apparatus for selecting a menu option associated with an overshot selectable region on the 
display, said apparatus comprising: 

(a) means for displaying a plurality of selectable regions within a first polygon 
intersecting the display, each selectable region associated respectively with a menu 
option, each selectable region adjacent a side of the first polygon and the plurality of 
selectable regions together at least partially circumscribing a region on the display; 

(b) movement related signal receiving means for receiving the movement related signal 
indicating the successive locations; and 

(c) control means for 

( 1 ) moving the first cursor within the first polygon responsive to the successive 
locations indicated b> the movement related signal; 

(2) confining at least part of the first cursor to the first polygon: and 
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(3) in response to a first quantity equalling or exceeding a predetermined quantit> , 
the first quantity being a function of the durations of one or more successive 
periods of intersection of the first cursor and one of the selectable regions, 
selecting the menu option associated with the intersected selectable region. 

The apparatus of claim 88 wherein the first polygon is located on the display. 

The apparatus of claim 89 wherein at least one of the selectable regions intersects the at least 
partially circumscribed region. 

The apparatus of claim 89 wherein the control means is further operative to confine at least 
part of the first cursor to a second polygon on the display. 

The apparatus of claim 91 wherein the control means is further operative to switch, 
responsive to an intersection of the first cursor and one of the selectable regions, from 
confining at least part of the first cursor to the first polygon to confining at least pan of the 
first cursor to the second polygon . 

The apparatus of claim 91 wherein the control means is further operative to switch, 
responsive to a distance between two of the successive locations, from confining at least part 
of the first cursor to the first polygon to confining at least part of the first cursor to the 
second polygon 

The apparatus of claim 91 wherein the control means is further operative to switch, 
responsive to an angle indicated by three of the successive locations, from confining at least 
part of the first cursor to the first polygon to confining at least part of the first cursor to the 
second polygon 

The apparatus of claim 91 wherein the first polygon intersects the second polygon. 

The apparatus of claim 95 wherein the first polygon includes all of the area of the second 
polygon. 
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97. The apparatus of claim 89 wherein the selection of the control means is further responsive to 
the proximity of one of the successive locations indicated by the movement related signal to 
the location of the first cursor. 

98. The apparatus of claim 89 wherein the selection of the control means is further responsive to 
the proximity of one of the successive locations indicated by the movement related signal to 
the intersected selectable region. 

99. The apparatus of claim S9 wherein the first polygon has at least five sides. 

100. The apparatus of claim 89 wherein at least one of the plurality of selectable regions is 
associated with an icon on the display. 

101. The apparatus of claim 100 wherein the icon represents one of a sign of a manual sign 

15 language, a location relative to a human body, a movement of a manual sign language a topic 

of conversation, a sentence, a desired direction of movement of a second cursor on the ~ 
display, a sequence of one or more graphics including an ideograph, and a symbol of a 
symbol set. 

20 102. An apparatus for selecting a submenu option from a menu hierarchy, said apparatus 
comprising: 

(a) a display area; 

(b) a menu comprising a plural it> of menu options, at least one of the menu options 
associated with a submenu comprising a plurality of submenu options: 

25 (c) means for at least partially delimiting: 

(1) a plurality of first selectable regions, each of the first selectable regions 
associated respectively with one of the menu options and each of the first 
selectable regions including a first subregion adjacent the display area and a first 
subregion on the display area, the plurality of the first subregions on the display 

30 area together at least partially circumscribing a first region on the display area; 

and 

(2) a plurality of second selectable regions, each of the second selectable regions 
associated respectively with one of the submenu options and each of the second 
selectable regions including a second subregion adjacent the display area and a 
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second subregion on the display area, the plurality of the second subregions on 
the display area together at least partially circumscribing a second region on the 
display area: 

(d) movement related signal receiving means for receiving a movement related signal 
5 indicating successive locations with respect to the display area; and 

(e) selection means for selecting, in response to a first dwell event, the menu option 
associated with the first selectable region intersected by one of the successive locations 
indicated by the movement related signal, the menu option being one of the menu 
options associated with a submenu, and for selecting, in response to a second dwell 

10 event, the submenu option associated with the second selectable region intersected by 

one of the successive locations indicated by the movement related signal. 



103. The apparatus of claim 102 wherein one of the menu options represents a group of characters 
and wherein a first one of the submenu options represents a first one character of the group 
15 of characters. 



104. The apparatus of claim 103 wherein each character of the group of characters has one of: 
(a) an extension at least a predetermined distance above the baseline of the group of 
characters; 

20 (b) an extension below the baseline of the group of characters; 

(c) lack of the characteristic described in (a); and 

(d) lack of the characteristic described in (b). 



25 105. The apparatus of claim 103 wherein the distance on the display area between the first 

subregion on the display area associated with the menu option representing the group of 
characters and the second subregion on the display area associated with the submenu option 
representing the first one character of the group of characters is responsive to the frequency 
of use of the first one character. 

30 

106. The apparatus of claim 103 wherein: 

(a) a second one of the submenu options represents a second one character of the group of 
characters; 

(b) the first one character is more frequently used than the second one character: and 
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(c) the distance on the display area between the first subregion on the display area 

associated with the menu option representing the group of characters and the second 
subregion associated with the submenu option representing the first one character of 
the group of characters is less than the distance on the display area between the first 
subregion on the display area associated with the menu option representing the group 
of characters and the second subregion on the display area associated with the 
submenu option representing the second one character of the group of characters. 



107, The apparatus of claim 1 03 wherein the position of a character of the group of characters 
indicates the position of the second subregion on the display area associated with the 
submenu option representing the first one character of the group of characters. 



108. An apparatus for selecting a menu option from a pluralit>' of menu options, said apparatus 
comprising: 

(a) a surface; 

(b) means for delimiting a plurality of selectable regions on the surface, each of the 
selectable regions associated respectively with a menu option, the plurality of 
selectable regions together at least partially circumscribing a region on the surface; 

(c) a pointer, responsive to the movement of a one of an operator's limbs, digits and head, 
for indicating successive locations on the surface; and 

(d) selection means for selecting, in response to a dwell event, the menu option associated 
with the selectable region intersected by one of the successive locations indicated by 
the pointer. 

109. An apparatus for selecting a menu option from a plurality of menu options, said apparatus 
comprising: 

(a) means for displaying a plurality of selectable regions on a display area, each of the 
selectable regions associated respectively with a menu option, the plurality of 
selectable regions together at least partially circumscribing a region on the display 
area; 

(b) movement related signal receiving means for receiving a movement related signal - 
indicating successive locations on the surface: and 

(c) in response a quantity equalling or exceeding a predetermined quantit>. the quantit> 
being a function of the durations of a plurality of successive periods of intersection of 
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two or more of the successive locations and one of the selectable regions, selection 
means for selecting the menu option associated with the intersected selectable region. 

110. An apparatus for selecting an option from a menu, said apparatus comprising: 
5 (a) cursor movement means for receiving a movement related signal and for moving a 

cursor on a display responsive to the received movement signal; 

(b) delimit means for delimiting on the display a first plurality of regions and a second 
plurality of selectable regions, each of the second plurality of selectable regions 
associated respectively with a menu option; the first plurality of regions together at 

10 least partially circumscribing a first region on the display; and 

(c) selection means, responsive only to an intersection of the cursor and a first one of the 
first plurality of regions and thereafter to a first selection event associated with one of 
the second pluralit\ of selectable regions, for selecting the menu option associated with 
the selectable region associated with the first selection event. 

15 

111- The apparatus of claim 1 1 0 further comprising means for receiving a switch operation signal; 
and wherein the delimit means includes means for displaying the first plurality of regions 
responsive to the received switch operation signal. 

20 112. The apparatus of claim 1 10 wherein the second plurality of selectable regions together at 
least partially circumscribing the first region on the display. 

1 13 The apparatus of claim 1 10 further comprising a third plurality of selectable regions, each of 
the third plurality of selectable regions associated respectively with a menu option; and 
25 wherein the selection means is further responsive to an intersection of the cursor and a 

second one of the first plurality of regions and thereafter to a second selection event 
associated with one of the third plurality of selectable regions, for selecting the menu option 
associated with the selectable region associated with the second selection event. 

30 114. The apparatus of claim 1 10 wherein the selection means includes means for receiving a 
switch operation signal; and wherein the first selection event includes: 

(1 ) an intersection of the cursor and the selectable region associated with the second 
selection event; and 

(2) at or near the lime the intersection occurs, receipt of the sw itch operation signal. 
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115. in a human interface system wherein a body member of an operator may indicate successive 
locations with respect to a display, a menu option selector for selecting a menu option from a 
plurality of menu options, said menu option selector comprising: 

(a) the display having thereon a first plurality of selectable regions, each of the first 
plurality of selectable regions associated respectively with one of the menu options; 

(b) means for at least partially delimiting a second plurality of selectable regions, each of 
the second plurality of selectable regions located outside the display and each 
associated respectively with one of the first plurality of selectable region: 

(c) in response to a first quantity equalling or exceeding a predetermined quantir>\ the first 
quantity being a function of: 

( 1 ) the durations of one or more successive periods of intersection of two or more of 
the successive locations and one of the selectable regions on the display area: 
and 

(2) the durations of one or more successive periods of intersection of two or more of 
the successive locations and the selectable region outside the display area . 
associated with the one of the selectable regions on the display area; 

a selector for selecting the menu option associated with the one of the selectable 
regions on the display area. 

1 16- The apparatus of claim 1 1 5 wherein the plurality of the subregions on the display together at 
least partially circumscribing a region on the display. 

117. In a human interface system wherein a body member of an operator may indicate a location 
on a surface, a menu option selector comprising: 

(a) the surface including a display area, the display area having thereon a plurality of 
selectable regions, each of the selectable regions associated respectively with a menu 
option, the plurality of selectable regions together at least partially circumscribing a 
region on the display area; 

(b) a clipper for generating, in response to the location indicated by the body member of 
the operator indicating a location outside the display area, a clipped location indicative 
of a location on the display area: and 

(c) a selector for selecting, in response to a selection event, the menu option associated 
with the selectable region intersected by the clipped location. 
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In a human interface system wherein a body member of an operator may indicate a location 
on a surface, a menu option selector comprising: 

(a) the surface including a display area, the display area having thereon a plurality of 
selectable regions, each of the selectable regions associated respectively with a menu 
option, the plurality of selectable regions together at least partially circumscribing a 
region on the display area: 

(b) a confmer for confming the location indicated by the body member of the operator to 
the display area: and 

(c) a selector for selecting, in response to a selection event, the menu option associated 
with the selectable region intersected by the location indicated by the body member of 
the operator. 

The menu option selector of claim 1 1 7 or 118 wherein each of the plurality of selectable 
regions is adjacent an edge of the display area. 

In a human interface system wherein a body member of an operator may indicate successive 
locations on a surface, a menu option selector comprising: 

(a) a detector area on the surface and including a plurality of selectable regions, each of 
the selectable regions associated respectively with a menu option; 

(b) a confmer for confining the location indicated by the body member of the operator to 
the detector area; and 

(c) a selector for selecting, in response to a dwell event associated with any one of the 
selectable regions, the menu option associated with the selectable region associated 
with the dwell event. 

The menu option selector of claim 120 wherein each of the plurality of selectable regions is 
adjacent an edge of the detector area. 

For use with a general purpose computer system including a display on which a cursor may 
be displayed, the general purpose computer system being capable of executing an application 
program, an apparatus comprising: 

(a) a medium readable by the general purpose computer system; and 
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(b) a program, stored on the medium and executable by the general purpose computer 
system, for: 

( 1 ) displaying a plurality of selectable regions within a polygon on the display, each 
selectable region adjacent a side of the polygon, one or more of the selectable 
regions each associated respectively with a sequence of one or more characters, 
the plurality of selectable regions together at least partially circumscribing a 
region on the display; 

(2) receiving a movement related signal and moving at least part of the cursor only 
within the polygon responsive to the movement related signal; and 

(3) in response to a first quantit>' equalling or exceeding a predetermined quantity, 
the first quantity being a function of the durations of one or more successive 
periods of intersection of the cursor and one of the one or more selectable 
regions, inputting the sequence of one or more characters associated with the 
intersected selectable region to the application program. 



123. A data entry system including a computer system on which may be executed an application 
program, said data entry comprising: 

(a) the computer system including a display; 

(b) a pointer selected from the group consisting of a ( 1 ) mouse; (2) trackball; (3) joystick; 
20 (4) stylus and graphics tablet; (5) lightpen; (6) thumb wheel; (7) touch screen; (8) 

head pointer; and (9) intraoral pointer, the pointer coupled to the computer 
system; and 

(c) program means executable on the computer system for: 

( 1 ) displaying a plurality' of selectable regions within a polygon on the display, each 
25 selectable region adjacent a side of the polygon, the plurality of selectable 

regions together at least partially circumscribing a region on the display; 

(2) moving a cursor within the polygon responsive to movement of the pointer; and 

(3) in response to a selection event and an intersection of the cursor and a selectable 
region associated with an input for the application program, inpuning the input 

30 to the application program. 

124. A computer access system for an operator having impaired motor capability', said computer 
access system including a computer system on which may be executed a computer program, 
said computer access system comprising: 
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(a) the computer system including a display: 

(b) program means executable on the computer system for: 

( 1 ) displaying a plurality of selectable regions within a polygon on the display, each 
5 selectable region adjacent a side of the polygon, the plurality' of selectable 

regions together at least partially circumscribing a region on the display; 

(2) receiving a movement related signal and moving at least part of a cursor only 
within the polygon responsive to the movement related signal: and 

(3) in response to a selection event and an intersection of the cursor and a selectable 
10 region associated with an input for the computer program, inpuning the input to 

the computer program. 

125. A speech synthesis system comprising: 

(a) a display on which may be displayed a plurality of selectable regions within a polygon 
15 on the display, each selectable region adjacent a side of the polygon and one or more 

of the selectable regions associated respectively with a sequence of one or more 
characters, the plurality of selectable regions together at least partially circumscribing 
a region on the display; 

(b) a speech synthesizer; and 
20 (c) control means for: 

( 1 ) receiving a movement related signal and moving a cursor within the polygon 
responsive to the movement related signal; 

(2) repetitively, in response to a selection event and an intersection of the cursor and 
one of the selectable regions associated with one of the one or more sequences 

25 of one or more letters, appending the sequence associated with the intersected 

selectable region to at least one previously selected sequence: and 

(3) speaking, by means of the speech synthesizer, the word spelled by the appended 
sequences. 

30 126. A device controller comprising: 

(a) means for displaying a plurality of selectable regions within a polygon on a surface, 
each selectable region adjacent a side of the polygon and each selectable region 
associated respectively with a device control function, the plurality of selectable 
regions together at least panially circumscribing a region of the polygon: 
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(b) signal generating means coupled to a device for generating a device control signal: and 

(c) control means for: 

( 1 ) receiving a movement related signal and moving at least part of a cursor only 
within the polygon in response to the received movement related signal; and 

(2) in response to a selection event, generating a device control signal corresponding 
to the device control function associated with the one of the plurality of 
selectable regions intersected by the cursor. 

127. The device controller of claim 126 wherein the device includes any one of a wheelchair, a 
household appliance, an appliance for use in an office, a workstation, a robot, and a computer 
peripheral. 

128. For use with a surface comprising a display area, a method of selecting a menu option from^a 
plurality of menu options, said method comprising the steps of: 

at least partially delimiting a plurality of selectable regions, each of the selectable 
regions associated respectively with a menu option and each of the selectable regions 
including an invisible subregion outside the display area and a visible subregion on the 
display area, the plurality of visible subregions together at least partially 
circumscribing a region on the display area; 

receiving a movement related signal indicating successive locations with respect to the 
display area: and 

selecting, in response to a dwell event associated with one of the selectable regions, the 
menu option associated with the selectable region associated with the dwell event. 

129. For use with a human interface system wherein a body member of an operator may indicate 
successive locations on a surface, the surface including a display area, the display area 
having thereon a plurality of selectable regions, each of the selectable regions associated 
respectively with a menu option and the plurality of selectable regions together at least 
partially circumscribing a region on the display area, a method of selecting a menu option 
from a plurality of menu options, said method comprising the steps of: 
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confining each of the successive locations to the display area: and 

selecting, in response to a dwell event associated with one of the selectable regions, the 
menu option associated with the selectable region associated with the dwell event. 

130- A method of speaking using a speech synthesis system including a display and a speech 
synthesizer, said method comprising the steps of: 

displaying a plurality of selectable regions within a polygon on the display, each 
10 selectable region adjacent a side of the polygon and one or more of the selectable 

regions associated respectively with a sequence of one or more characters, the pluraiit>' 
of selectable regions together at least partially circumscribing a region on the display; 

receiving a movement related signal and moving at least part of a cursor only within 
15 the polygon responsive to the movement related signal; 

repetitively: 

(i) in response to a first quantity equalling or exceeding a predetermined quantity, 
20 the first quantity being a function of the durations of one or more successive 

periods of intersection of the cursor and one of the one or more selectable 
regions, selecting the sequence associated with the intersected selectable region; 
and 

25 (ii) appending the selected sequence to at least one previously selected sequence; 

and 

speaking, by means of the speech synthesizer, the word spelled by the appended 
sequences. 
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131. A method of inputting dau to a computer program for an operator having impaired motor 
capabilit\', said method comprising the steps of: 

displaying a plurality of selectable regions within a polygon on a display, each 
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selectable region adjacent a side of the polygon and each selectable region associated 
respectively with an input for the computer program, the plurality of selectable regions 
together at least partially circumscribing a region on the display: 

receiving a movement related signal and moving at least part of a cursor only within 
the polygon responsive to the movement related signal; and 

in response to a first quantit> equalling or exceeding a predetermined quantir> , the first 
quantit\' being a function of the durations of one or more successive periods of 
intersection of the cursor and one of the selectable regions, inputting the input 
associated with the intersected selectable region to the computer program. 

132. An apparatus for indicating dwell lime comprising: 

(a) a surface with respect to which a region may be at least partially delimited; 

(b) movement receiving means for receiving a movement related signal indicating a first 
location intersecting the region and, at a later time, a second location intersecting the 
region; and 

(c) indicating means for indicating at least the difference between the time the second 
location occurs and the time the first location occurs. 

133. The apparatus of claim 132 wherein the indicating means includes means for determining the 
difference between the time the second location occurs and the time the first location occurs. 

134. The apparatus of claim 132 wherein the indicating means includes an indicator and means for 
modifying the appearance of the indicator. 

135. The apparatus of claim 132 wherein the region intersects the surface. 

136. The apparatus of claim 1 32 wherein the region is adjacent the surface. 

137. The apparatus of claim 132 wherein the movement related signal further indicates, at a time 
later than the time the second location occurs, a third location not intersecting the region and 
the indicating means is further operative to indicate the non-intersection of the third location 
and the region. 
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The apparatus of claim 132 wherein the movement related signal further indicates, at a time 
later than the lime the second location occurs, a third location not intersecting the region and, 
at a later time, indicates a fourth location not intersecting the region and the indicating means 
is further operative to indicate at least the difference between the time the fourth location 
occurs and the time the third location occurs. 

The apparatus of claim 132 wherein the indicating means includes means for producing an 
output signal which varies in at least one way as the difference between the time the second 
location occurs and the time the first location occurs increases and varies in at least the 
opposite way as the difference between the time the fourth location occurs and the time the 
third location occurs increases. 

An apparatus for indicating dwell time comprising: 

(a) a pointer; 

(b) a detector; and 

(c) an indicator for indicating at least the duration of a period the pointer indicates the 



The apparatus of claim 140 wherein the indication of the duration of the period of detection 
includes a visible signal. 

The apparatus of claim 140 wherein the indication of the duration of the period of detection 
includes an audible signal. 

The apparatus of claim 140 wherein the indication of the duration of the period of detection 
includes a tactile signal. 

An apparatus for indicating the remaining dwell time required to select a selectable region, 
said apparatus comprising: 

(a) a surface with respect to which the selectable region may be at least partially 
delimited: 

(b) movement receiving means for receiving a movement related signal indicating 
successive locations with respect to the surface; and 
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(c) indicating means for modifying in a first manner the appearance of a first indicating 
region on the surface responsive to a period of intersection of two or more of the 
successive locations and the selectable region not equalling or exceeding a 
predetermined period and for modifying in a second manner the appearance of a 
second indicating region on the surface responsive to a period of intersection of two or 
more of the successive locations and the selectable region equalling or exceeding the 
predetermined period. 

The apparatus of claim 144 wherein the modification in the first manner includes a slight 
modification and the modification in the second manner includes a marked modification. 

The apparatus of claim 144 wherein the modification in the first manner includes a 
modification in brightness. 

The apparatus of claim 144 wherein the modification in the second manner includes a 
modification in hue. 



The apparatus of claim 144 wherein the first indicating region intersects the second 
indicating region. 

The apparatus of claim 144 wherein one of the first and second indicating regions intersects 
the selectable region. 

The apparatus of claim 144 wherein one of the first and second indicating regions is 
coterminous with the selectable region. 

An apparatus for indicating dwell time comprising: 

(a) a surface with respect to which a plurality of regions may be at least partially 
delimited; 

(b) movement receiving means for receiving a movement related signal indicating a first 
location intersecting any one of the plurality of regions and. at a later time, a second 
location intersecting the region intersected by the first location: and 

(c) a plurality of indicators, each associated respectively with a region, for indicating at 
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least the difference between the time the second location occurs and the time the first 
location occurs. 

1 52. The apparatus of claim 1 5 1 wherein each indicator is adjacent its associated region. 

153. The apparatus of claim 151 wherein each indicator intersects its associated region. 

1 54. An apparatus for selecting a menu option from a pluralit\ of menu options, said apparatus 
comprising: 

(a) a surface with respect to which a pluralit>' of selectable regions may be at least 
partially delimited, each selectable region associated respectively with a menu option; 

(b) movement receiving means for receiving a movement related signal indicating 
successive locations with respect to the surface: and 

(c) selection means for selecting, in response to a dynamic dwell event, the menu option 
associated with the selectable region intersected by the location of the successive 
locations whose occurrence triggers the dynamic dwell event. 

155. In a human interface system wherein a body member of an operator may indicate successive 
locations with respect to a surface, a menu option selector comprising: 

(a) the surface with respect to which a selectable region may be at least partially 
delimited, the selectable region associated with a menu option; 

(b) a detector for detecting the successive locations; and 

(c) responsive to: 

( 1 ) the duration of a first period during which two or more of the successive 
locations intersect the selectable region: 

(2) the duration of a second period, occurring after the first period, during which two 
or more of the successive locations do not intersect the selectable region; and 

(3) the duration of a third period, occurring after the second period, during which 
two or more of the successive locations intersect the selectable region; 

a selector for selecting the menu option. 

1 56. A device controller comprising: 

(a) display means for displaying a pluralit\ of selectable regions on a surface, each 
selectable region associated respectively with a device control function: 
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(b) movement receiving means for receiving a movement related signal indicating 
successive locations on the surface; 

(c) indicating means for indicating at least the duration of a period of intersection of two 
or more of the successive locations indicated by the movement related signal and a 
selectable region; and 

(d) initiation means, responsive to a dwell event, for initiating the control function 
associated with the intersected selectable region. 

The device controller of claim 1 56 further comprising a controlled device, responsive to the 
initiation means, for performing the device control function. 

The device controller of claim 1 57 wherein the controlled device includes one of a household 
appliance and a motorized wheelchair. 

A computer access device for an operator having impaired motor control, said device 
comprising: 

(a) a computer system, including a display on which my be displayed a plurality of 
selectable regions, each of the selectable regions associated respectively with a 
sequence of one or more characters, and on which display a moveable cursor may be 
displayed and then moved responsive to a movement related signal, the computer 
system being further capable of executing an application program operative to accept 
signals representing any of the plurality of sequences of one or more characters; and 

(b) an access program, for execution on the computer system, for indicating at least the 
duration of a period of intersection of the cursor and any one of the one or more 
selectable regions and for inputting to the application program signals representing the 
sequence of one or more characters associated with the intersected selectable region in 
response to the period of intersection of the cursor and the intersected selectable region 
equalling or exceeding a predetermined period. 

For use with a general purpose computer system including a display with respect to which 
may be at least partially delimited a plurality of selectable regions, each of the selectable 
regions associated respectively with a sequence of one or more characters, wherein the 
general purpose computer system is capable of receiving a movement related signal 
indicating successive locations with respect to the display, and wherein the genera! purpose 
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computer system is capable of executing an application program: 

(a) a medium readable by the general purpose computer system; and 

(b) a program, stored on the medium and executable by the general purpose computer 
system, for: 

( 1 ) indicating at least the duration of a period of intersection of two or more of the 
successive locations and any one of the selectable regions: and 

(2) passing to the application program the sequence of one or more characters 
associated with the intersected selectable region in response to the duration of 
the period of intersection equalling or exceeding a predetermined period. 



10 



161. For use with a processing unit, 

(a) said processing unit coupled to a display on which may be displayed a plurality of 

selectable regions, each region associated respectively with a sequence of one or more 
characters: 

15 (b) said processing unit being capable of receiving a movement related signal indicating 

successive locations on the display; 
(c) said processing unit being capable of executing an application program; 

an access program, executable on the processing unit, for an operator having impaired motor 
20 control, said access program operative to indicate at least the duration of a period of 

intersection of two or more of the successive locations and an intersected selectable region 
and to pass to the application program the sequence of one or more characters associated 
wr ith the intersected selectable region in response to the duration of the period of intersection 
equalling or exceeding a predetermined period. 

25 

162. The computer access device of claim 161 wherein the access program and the application 
program include at least some common code. 

163. For use with a computer system, a computer access device for an operator having impaired 
30 motor control, said device comprising: 

(a) a pointer; 

(b) a detector associated with a sequence of one or more characters; 

(c) an indicator for indicating at least the duration of a period the pointer indicates the 
detector; and 
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(d) a processing unit coupled to the computer system for inputting to the computer 

system, in response to a dwell event, signals representing the sequence of one or more 
characters associated with the detector. 

5 164, A speech synthesis system comprising: 

(a) a display with respect to which may be delimited a plurality of selectable regions, each 
of the selectable regions associated respectively with a sequence of one or more 
characters: 

(b) a speech synthesizer: and 
10 (c) control means for 

( 1 ) receiving a movement related signal indicating successive locations with respect 
to the surface; 

(2) repetitively 

(a ) indicating the duration of a first period of intersection of two or more of 
15 the successive locations and a first one of the selectable regions: and 

(b) in response to the duration of a second period of intersection of two or 

more of the successive locations and the first one of the selectable regions 
equalling or exceeding a predetermined period, appending the sequence 
associated with the first one of the selectable regions to at least one 
20 previously selected sequence; and 

(3) speaking, by means of the speech synthesizer, the word spelled by the appended 
sequences. 

165, A speech synthesis system comprising: 
25 (a) a display on which may be displayed a pluralit>' of selectable regions, each selectable 

region associated respectively with a sequence of one or more characters; 

(b) a speech synthesizer; and 

(c) control means for: 

(1) receiving a movement related signal indicating: 
30 (A) a first location intersecting any one of the plurality of selectable regions; 

(B) at a time after the first location occurs, a second location intersecting the 
selectable region intersected by the first location: 

(C) at a time after the second location occurs, a third location not intersecting 
the selectable region intersected by the first location: 
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(D) at a time after the third location occurs, a fourth location not intersecting 
the selectable region intersected by the first location: 

(E) at a time after the fourth location occurs, a fifth location intersecting the 
selectable region intersected by the first location; and 

5 (F) at a time after the fifth location occurs, a sixth location intersecting the 

selectable region intersected by the first location: 
(2) repetitively: 

(A) responsive to the difference between the lime the second location occurs 
and the time the first location occurs, increasing the brightness of the 

10 selectable region: 

(B) responsive to the difference between the time the fourth location occurs 
and the time the third location occurs, decreasing the brightness of the 
selectable region: and 

(C) responsive to a quantity that is a function of: 

15 (i) the difference between the time the second location occurs and the 

time the first location occurs; 

(ii) the difference between the time the fourth location occurs and the 
time the second location occurs; and 

(iii) the difference between the time the sixth location occurs and the 
20 time the fifth location occurs; 

equalling or exceeding a predetermined quantit>', appending the sequence 
associated with the selectable region to at least one previously selected 
sequence: and 

25 (3) speaking, by means of the speech synthesizer, the word spelled by the appended 

sequences. 

166. A data entry system comprising: 

(a) a display having thereon a plurality of selectable data entry items and a region for 
30 indicating the number of units of the data entry item selected; 

(b) a plurality of detectors, each associated respectively with one of the selectable data 
entr\' items; 

(c) a pointer for emitting signals directed at the data entr\' items: and 

(d) control means for controlling the operation of said data entr\ system, the control 
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means including: 

( I ) first indicating means for producing signals in resf>onse to the signals falling on 
the detectors, said first indicating means giving a first indication of the period of 
time that the pointer has been directed to a first one of the data entry items; 
5 (2) second indicating means for producing signals in response to the signals falling 

on the detectors, said second indicating means giving a second indication that 
the pointer has been directed to a second one of the data entr\^ items for at least a 
predetermined period of time and that the second one of the data entr\ items is 
selected for entr\ into the data entr\' system: and 
10 (3) third indicating means for indicating in the region the number of units of the 

data entry item selected. 

167. A order entry system comprising: 

(a) a display having thereon a pluralit\ of selectable order entry items, a first region for 
15 indicating the number of units of the order entry item selected and a second region for 

indicating the total monetary value of the. units selected; 

(b) a plurality of detectors, each associated respectively with one of the selectable order 
entry items; 

(c) a pointer for emitting signals directed at the order entry items; and 

20 (d) control means for controlling the operation of said order entry system, the control 

means including 

( 1 ) first indicating means for producing signals in response to the signals falling on 
the detectors, said first indicating means giving a first indication of the duration 
of the period that the pointer has been directed to a first one of the order entrv 

25 items; 

(2) second indicating means for producing signals in response to the signals falling 
on the detectors, said second indicating means giving a second indication that 
the pointer has been directed to a second one of the order entry items for at least 
a predetermined period of time and that the second one of the order entry items 

30 is selected for entry into the order entr>' system; 

(3) third indicating means for indicating in the first region the number of units of the 
order entry item selected; 

(4) totaling means for totaling the monetar>' value of the units selected; and 

(5) fourth indicating means for indicating in the second region the total monetar> 
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The order entry system of claim 167 wherein the first indicating means further includes third 
indicating means for giving a third indication of the duration of the period that the pointer is 
not directed to an order entr>' item to which it was previously directed. 

The order entrv' system of claim 167 wherein the display further includes a pictorial 
representation for at least some of the order entrv' items and wherein the detectors are located 
within each pictorial representation. 

The order entry system of claim 169 wherein the pictorial representation is in the form of a 
food item at a fast- food restaurant. 

The order enir\' system of claim 167 wherein the pointer includes: 

(a) an infrared light source; and 

(b) a lens for producing a narrow beam of light in cooperation with the infrared light 



The order entry system of claim 167 further comprising a stationary housing for housing the 
pointer, said housing positioned relative to the display to enable a person using the order 
entry system and seated in a vehicle to pick up the pointer, use it, and return it to the 
stationary housing while seated in the vehicle. 

An apparatus for selecting a menu option from a plurality of menu options, said apparatus 
comprising: 

(a) a surface including a plurality of detectors each associated respectively with a menu 
option; 

(b) a pointer operative to emit energy detectable by each of the plurality of detectors; and 

(c) responsive to: 

( 1 ) the duration of a first period during which a particular detector detects energy 
emitted by the pointer above a first threshold; 

(2) the duration of a second period, occurring after the first period, during which the 
particular detector does not detect energy emined by the pointer above a second 
threshold; and 
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(3) the duration of a third period, occurring after the second period, during which the 

particular detector detects energy emined by the pointer above a third threshold; 
a selector for selecting the menu option associated with the particular detector. 

5 1 74. The apparatus of claim 1 73 wherein the selector selects the menu option associated with the 
particular detector in response to the sum of a multiple of the duration of the first period and 
a multiple of the duration of the third period minus a multiple of the duration of the second 
period equalling or exceeding a predetermined quantity, 

10 175. An apparatus for inpuning words, said apparatus comprising: 

(a) a display with respect to which may be delimited a selectable region associated with a 
word; 

(b) word receiving means for receiving a signal representing the word: and 

(c) control means for: 

15 ( 1 ) receiving a movement related signal indicating successive locations with respect 

to the surface; 

(2) indicating the duration of a period of intersection of two or more of the 
successive locations and the selectable region; and 

(3) inputting, in response to the period of intersection equalling or exceeding a 
20 predetermined period, the signal representing the word to the word receiving 

means. 

1 76. The apparatus of claim 1 75 wherein the word belongs to an ideographic language. 

25 1 77. A method of indicating the remaining dwell time required to select a selectable region, said 
method comprising the steps of: 

displaying the selectable region on a surface; 

30 receiving a movement related signal indicating a first location intersecting the selectable 

region and, at a time after the first location occurs, a second location intersecting the 
selectable region; and 

indicating the difference between the time the second location occurs and the time the first 
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location occurs. 

The method of claim 1 77 further comprising the steps of: 

receiving a movement related signal indicating, at a time after the lime the second location 
occurs, a third location not intersecting the selectable region; and 

indicating the non-intersection of the third location and the selectable region. 

The method of claim 1 78 further comprising the steps of: 

receiving a movement related signal indicating, at a time after the time the third location 
occurs, a fourth location not intersecting the selectable region: and 

indicating the difference between the time the fourth location occurs and the time the third 
location occurs. 

In a human interface system including a surface on which may be displayed a cursor whose 
movement is responsive to a first movement related signal, and with respect to which surface 
may be at least partially delimited a plurality of selectable regions, each of the plurality of 
selectable regions associated respectively with a menu option, an apparatus for selecting any 
one of the plurality of menu options, said apparatus comprising: 

(a) movement related signal receiving means for receiving the first movement related 
signal: and 

(b) selection means for selecting the menu option associated with the particular one of the 
plurality of selectable regions most nearly along a first cursor path indicated by a first 
cursor location and the first movement related signal in advance of an intersection of 
the cursor and the particular selectable region. 

The apparatus of claim 180 wherein the movement related signal receiving means is ftjrther 
operative to receive a second movement related signal occurring subsequent to the first 
movement related signal; wherein the selection means is further operative to select the menu 
option associated with the specific one of the pluralit> of selectable regions most nearK 
along a second cursor path indicated by a second cursor location and the second movement 
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related signal in advance of an intersection of the cursor and the specific selectable region; 
wherein the first and second cursor paths are parallel: and wherein the particular selectable 
region is not the specific selectable region, 

182. In a human interface system including a surface on which may be displayed a cursor whose 
movement is responsive to a first movement related signal, and with respect to which surface 
may be at least partially delimited a pluralit>' of selectable regions, each of the plurality of 
selectable regions associated respectively with a menu option, an apparatus for selecting any 
one of the plurality of menu options, said apparatus comprising: 

(a) movement related signal receiving means for receiving the first movement related 
signal; and 

(b) selection means for selecting the menu option associated with the particular one of the 
plurality of selectable regions most nearly along a first cursor path in response to an 
intersection of the cursor and said particular selectable region. 

183. The apparatus of claim 180 or 182 further comprising means for indicating which one of the 
plurality of selectable regions is most nearly along the first cursor path. 

184. The apparatus of claim 1 80 or 1 82 wherein the surface includes a display area and each of 
the plurality of selectable regions is adjacent an edge of the display area. 

1 85. The apparatus of claim 1 80 or 1 82 wherein the plurality of selectable regions together at least 
partially circumscribes an area on the surface. 

1 86. An apparatus for selecting an option from a menu, said apparatus comprising: 

(a) means for at least partially delimiting a plurality of selectable regions with respect to a 
surface, each of the plurality of selectable regions associated respectively with a menu 
option; 

(b) movement related signal receiving means for receiving a movement related signal: and 

(c) control means for: 

(1 ) moving a cursor on the surface in response to the received movement related 
signal; 

(2) changing a selection threshold period responsive to a cursor path; 

(3 ) delecting an intersection of the cursor and any one of the plurality of selectable 
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regions; and 

(4) in response to the period of the intersection equalling or exceeding the selection 
threshold period, selecting the menu option associated with the intersected 
selectable region. 

5 

187. The apparatus of claim 186 wherein the control means is further operative to indicate the 
changed length of the selection threshold period. 



1 88. The apparatus of claim 1 86 wherein one of the plurality of selectable regions has previously 
10 been selected and the amount of change to the selection threshold period varies in response 

to which one of the selectable regions has been previously selected. 



189. The apparatus of claim 1 86 wherein the changed selection threshold period equals or exceeds 
a predetermined period. 

15 

190. The apparatus of claim 1 86 wherein one of the plurality of selectable regions has previously 
been selected and each location of the cursor in the cursor path is outside the previously 
selected selectable region. 



20 191 . The apparatus of claim 190 wherein each location of the cursor in the cursor path is at least a 
predetermined distance from the previously selected selectable region. 

192. The apparatus of claim 190 wherein each location of the cursor in the cursor path occurs at 
least a predetermined period of time after the selection of the previously selected selectable 

25 region. 

193. The apparatus of claim 1 86 wherein the amount of change to the selection threshold period 
varies in response to the distance between the intersected selectable region and the location 
of the cursor. 

30 

194. The apparatus of claim 186 wherein the amount of change to the selection threshold period 
varies in response to the difference between the time the intersection occurs and the time the 
location of the cursor occurs. 
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The apparatus of claim 186 wherein each of the plurality of selectable regions includes a 
subregion visible on the surface and an invisible subregion located outside the surface and 
wherein the plurality of visible subregions together at least partially circumscribes an area on 
the surface. 

The apparatus of claim 186 wherein the movement related signal is responsive to the 
movement of a body member of an operator having impaired ability to sense the position of 
the body member and the apparatus further comprises tactile indication means for indicating 
with the body member tactilely to the operator the position of the body member. 

The apparatus of claim 1 86 wherein one or more of the plurality of menu options represents 

any one of: 

(a) a prefi.x: 

(b an infix: and 

(c) a suffix; 

to be added to a root word. 

An apparatus for selecting a menu option from a plurality of menu options, said apparatus 
comprising: 

(a) means for at least partially delimiting a plurality of selectable regions with respect to a 
surface, each of the plurality of selectable regions associated respectively with a menu 
option; 

(b) movement related signal receiving means for receiving a movement related signal 
indicating successive locations on the surface; and 

(c) control means for: 

( 1 ) moving a cursor in a first manner on the surface in response to the movement 
related signal and thereafter detecting an intersection of the cursor and any one 
of the plurality of selectable regions; 

(2) responsive to a first cursor path, moving the cursor in a second manner on the 
surface in response to the movement related signal, wherein the movement of the 
cursor in the second manner is attenuated relative to the movement of the cursor 
in the first manner; and 

(3) in response to the period of the intersection equalling or exceeding a selection 
threshold period, selecting the menu option associated with the intersected 



155 



BNSDOCID:<WO 9630822A1> 



wo 96/30S22 PCT/US95/03591 

selectable region, 

199. The apparatus of claim 198 wherein the moving of the cursor in the second manner is further 
responsive to the intersection. 

5 

200. The apparatus of claim 199 wherein the moving of the cursor in the second manner is further 
responsive to a path indicated by successive locations occurring after the intersection. 

201 . The apparatus of claim 199 wherein the moving of the cursor in the second manner is further 
10 responsive to an angle of attack of the cursor path to the intersected selectable region, 

202. The apparatus of claim 198 wherein the plurality of selectable regions together at least 
partially circumscribes an area on the surface. 

15 203. The apparatus of claim 198 wherein the moving of the cursor in the second manner is further 
responsive to the velocity of the first cursor path. 

204. The apparatus of claim 198 wherein the moving of the cursor in the second manner is further 
responsive to the acceleration of the first cursor path. 

20 

205. The apparatus of claim 198 wherein the moving of the cursor in the second manner is further 
responsive to the proximity of a cursor location in the first cursor path to the selectable 
region most nearly along the first cursor path. 

25 206. In a human interface system including a first surface on which may be displayed a pluralit> 
of selectable regions each associated respectively with a menu option, and wherein a body 
member of an operator may indicate successive locations on a second surface, each location 
corresponding to a location on the first surface, a selection controller comprising: 

(a) a direction detector for determining a move direction of the body member responsive 
30 to two or more of the successive locations on the second surface; and 

(b) a selector for selecting the menu option associated with the particular one of the 
plurality of selectable regions most nearly along a path on the first surface 
corresponding to the move direction of the body member, the selection occurring in 
advance of an intersection of anv one of the locations on the first surface and the 
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particular selectable region. 

207. In a human interface system including a first surface on which may be displayed a pluralit>' 
of selectable regions each associated respectively with a menu option, and wherein a body 
member of an operator may indicate successive locations on a second surface, each location 
corresponding to a location on the first surface, a selection controller comprising: 

(a) a direction detector for determining a move direction of the body member responsive 
to two or more of the successive locations on the second surface: and 

(b) a selector for selecting the menu option associated with the particular one of the 
plurality of selectable regions most nearly along a path on the first surface 
corresponding to the move direction of the body member, the selector being responsive 
to an intersection of one of the locations on the first surface and the particular 
selectable region. 

208. In a human interface system including 

a first surface on which may be displayed a plurality of regions each associated 
respectively with a menu option; 

wherein a body member of an operator may indicate successive locations on a second 
surface, each location corresponding to a location on the first surface; and 
wherein a particular one of the regions may be selected responsive to a period of 
intersection of two or more of the locations on the first surface and the particular 
region equalling or exceeding a selection threshold period; 
a selection threshold controller comprising: 

(a) a direction detector for determining a move direction of the body member responsive 
to two or more of the locations on the second surface; and 

(b) means for changing the selection threshold period responsive to the move direction of 
the body member. 

209. The selection threshold controller of claim 208 further comprising a pointer for indicating 
with the body member successive locations on the second surface. 

210. The selection threshold controller of claim 208 wherein the first surface includes the second 
surface. 
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211- The selection threshold controller of claim 208 wherein the second surface includes the first 
surface. 

212. The selection threshold controller of claim 208 wherein the body member of the operator 
includes any one of the operator's shoulder, the operator's arm, the operator's elbow, the 
operator's wrist, the operator's hand, the operator's finger, the operator's thumb, the operator's 
leg, the operator's knee, the operator's ankle, the operator's foot, the operator's toe. the 
operator's hip, the operator's trunk, the operator's neck, the operator's tongue, the operator's 
lip. the operator's eye and the operator's head. 

213. The selection threshold controller of claim 208 wherein the means for changing of the 
selection threshold period includes means for decreasing the selection threshold period 
responsive to the move direction indicating with the body member a path toward the 
particular region. 

214. The selection threshold controller of claim 208 wherein the means for changing of the 
selection threshold period includes means for increasing the selection threshold period 
responsive to the move direction indicating with the body member a path away from the 
particular region. 



215. In a human interface system including 

a first surface on which may be displayed a plurality of regions each associated 
respectively with a menu option: 

wherein a body member of an operator may indicate successive locations on a second 
25 surface, each location corresponding to a location on the first surface; and 

wherein a particular one of the regions may be selected responsive to a period of 

intersection of two or more of the locations on the first surface and the particular 

region equalling or exceeding a selection threshold period; 
a selection threshold controller comprising a threshold controller for changing the selection 
30 threshold period responsive to one of: 

(1) a path indicated by two or more of the successive locations on the first surface; and 

(2) one of the locations on the first surface intersecting a predetermined region on the first 
surface. 
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216. An apparatus for selecting an option from a menu, said apparatus for use with a general 
purpose computer system including a display on which may be displayed a cursor whose 
movement is responsive to a movement related signal, said apparatus comprising: 
(a) a medium readable by the general purpose computer system; and 
5 (b) an access program, stored on the medium and executable by the general purpose 

computer system, for: 

(1 ) at least partially delimiting a plurality of selectable regions with respect to the 
display, each of the plurality of selectable regions associated respectively with a 
menu option: and 

1 0 (2) selecting the menu option associated with the particular one of the plurality of 

selectable regions most nearly along a path indicated by a cursor location on the 
display and the movement related signal in advance of an intersection of the 
cursor and said particular selectable region. 

15 2 1 7. An apparatus for selecting an option from a menu, said apparatus for use with a general 
purpose computer system including a display on which may be'displayed a cursor whose 
movement is responsive to a movement related signal, said apparatus comprising: 

(a) a medium readable by the general purpose computer system; and 

(b) an access program, stored on the medium and executable by the general purpose 
20 computer system, for: 

( 1 ) at least partially delimiting a plurality of selectable regions with respect to the 
display, each of the plurality of selectable regions associated respectively with a 
menu option: and 

(2) selecting the menu option associated with the particular one of the plurality of 
25 selectable regions most nearly along a path indicated by a cursor location on the 

display and the movement related signal in response to an intersection of the 
cursor and said particular selectable region. 



218. An apparatus for selecting an option from a menu, said apparatus for use with a general 
30 purpose computer system including a display on which may be displayed a cursor whose 

movement is responsive to a movement related signal, said apparatus comprising: 

(a) a medium readable by the general purpose computer system: and 

(b) an access program, stored on the medium and executable by the general purpose 
computer system, for: 
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( 1 ) at least partially delimiting a plurality of selectable regions with respect to the 
display, each of the plurality of selectable regions associated respectively with a 
menu option; and 

(2) changing a selection threshold period responsive to a cursor path; 

5 (3) detecting an intersection of the cursor and any of the pluralit> of selectable 

regions; and 

(4) in response to the period of the intersection equalling or exceeding the selection 
threshold period, selecting the menu option associated with the intersected 
selectable region. 

10 

219. An apparatus including a surface, for measuring an individual's ability to move a position 
indicator on the surface in a direct path from a starting location on the surface to an ending 
location on the surface, said position indicator indicative of a position of one of the 
individual's body members, said apparatus comprising: 
15 (a) the surface; 

(b) position indicating with the body member means for indicating with the body member 
on the surface a position of the body member; 

(c) collection means for collecting data indicative of a succession of the indicated 
positions on the surface; and 

20 (d) measurement means for measuring the deviation of the path indicated by the collected 

data from a predetermined path between the starting location and the ending location. 



220. An interactive terminal, including a display on which may be displayed: 

a plurality of selectable regions, each of the plurality of selectable regions associated 
25 respectively with a menu option and a selection threshold period; and 

a cursor whose movement is responsive to the movement of a body member of an 
operator; 
said terminal comprising: 

(a) means for displaying the plurality of selectable regions on the display; and 
30 (b) control means for: 

(1 ) in response to cursor movement toward a particular one of the plurality of 
selectable regions, decreasing the selection threshold period associated with the 
particular selectable region; 

(2) in response to cursor movement diverging from a path toward the particular 
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(4) 



(3) 



selectable region, increasing the selection threshold period associated with the 
particular selectable region; 

detecting an intersection of the cursor and any of the plurality of selectable 
regions: and 

in response to the period of the intersection being equal to or greater than the 
selection threshold period associated with the intersected selectable region, 
selecting the menu option associated with the intersected selectable region. 



221 . The terminal of claim 220 wherein the pluralit>' of selectable regions together at least 



222. A data entry system comprising: 

(a) a computer system including: 

( 1 ) a display. on which a moveable cursor may be displayed and then moved 
responsive to a movement related signal and on which a plurality of selectable 
regions may be displayed, each selectable region associated respectively with a 
sequence of one or more characters and a character sequence signal representing 
the associated sequence of one or more characters; and 

(2) a processing unit coupled to the display and capable of executing an application 
program operative to accept any of the character sequence signals; and 

(b) an input program, for execution on the processing unit, for inputting to the application 
program the character sequence signal representing the sequence of one or more 
characters associated with the particular one of the plurality of selectable regions most 
nearly along the path indicated by a cursor location and the movement related signal in 
advance of an intersection of the cursor and said particular selectable region. 

223. A data entry system comprising: 

(a) a computer system including: 



( 1 ) a display on which a moveable cursor may be displayed and then moved 
responsive to a movement related signal and on which a plurality of selectable 
regions may be displayed, each selectable region associated respectively with a 
sequence of one or more characters; and 

(2) a processing unit coupled to the display and capable of executing an application 
program operative to accept any of the plurality of sequences of one or more 



10 



partially circumscribes an area on the display. 
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(b) an input program, for execution on the processing unit, for inputting to the application 
program the sequence of one or more characters associated with the particular one of 
the plurality of selectable regions most nearly along a cursor path, said input to the 
application program being responsive to an intersection of the cursor and said 
particular selectable region. 

The data entry system of claim 220 wherein the input program further comprises means for 
changing a selection threshold period associated with said particular selectable region and 
wherein said input to the application program is further responsive to the duration of the 
period of the intersection equalling or exceeding the selection threshold period. 

In a computer system including a display on which may be displayed a moveable cursor, and 
on which may be displayed a menu having a pluralitv' of menu options, each of the menu 
options associated respectively with a selectable region on the display, an apparatus for 
selecting any one of the plurality of menu options, said apparatus comprising: 

(a) identification means for identifying which one of the plurality of selectable regions is 
most nearly along a cursor path; and 

(b) selection means for selecting the menu option associated with the identified selectable 
region in advance of an intersection of the cursor and the identified selectable region. 

In a computer system including a display on which may be displayed a moveable cursor, and 
on which may be displayed a menu having a plurality of menu options, each of the menu 
options associated respectively with a selectable region on the display, an apparatus for 
selecting any one of the plurality of menu options, said apparatus comprising: 

(a) means for detecting an intersection of the cursor and any one of the selectable regions; 
and 

(b) selection means, responsive to the detected intersection, for selecting the menu option 
associated with the particular selectable region most nearly along a cursor path. 

The apparatus of claim 226 wherein the selection means further comprises means for 
decreasing a selection threshold period associated with the particular selectable region and 
wherein the selection means is further responsive to the duration of the period of the 
intersection equalling or exceeding the selection threshold period. 
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228. A speech synthesis system comprising: 

(a) a display on which may be displayed a plurality of selectable regions, each of the 

selectable regions associated respectively with a sequence of one or more characters; 
5 (b) a speech synthesizer; and 

(c) control means for: 

( 1 ) receiving a movement related signal and moving a cursor on the display 
responsive to the movement related signal; 

(2) repetitively 

10 (A) detecting an intersection of the cursor and any of the plurality of selectable 

regions; 

(B) changing a selection threshold period responsive to a cursor path to the 
intersected selectable region; and 

(C) in response to the period of the intersection equalling or exceeding the 
15 selection threshold period, appending the sequence associated with the 

intersected selectable region to at least one previously selected sequence; 
and 

(3) speaking, by means of the speech synthesizer, the word spelled by the appended 
sequences. 

20 

229. A device controller comprising: 

(a) means for displaying a plurality of selectable regions on a surface, each of the plurality 
of selectable regions associated respectively with a device control function; 

(b) movement related signal receiving means for receiving a movement related signal; 

25 (c) signal generating means coupled to a device for generating a device control signal; and 

(d) control means for: 

( 1 ) moving a cursor on the surface in response to the received movement related 
signal; 

(2) changing a selection threshold period responsive to a cursor path; and 

30 (3) detecting an intersection of the cursor and any one of the selectable regions; 

(4) in response to the period of the intersection equalling or exceeding the selection 
threshold period, generating a device control signal corresponding to the device 
control function associated with the intersected selectable region. 
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The controller of claim 229 wherein the device includes a wheelchair. 

In a computer system, an apparatus for measuring an individuars ability to move one of the 
individual's body members directly from a starting position to an ending position, said 
apparatus comprising: 

(a) receiving means, coupled to the computer system, for receiving a sampling among data 
indicative of successive positions of the body member: and 

(b) measurement means, executable on the computer system, for measuring deviation of 
the path indicated by the received position data from a direct path between the starting 
position and the ending position. 

The apparatus of claim 23 1 further comprising: 

(c) a display, coupled to the computer system, on which a moveable cursor may be 
displayed: and 

(d) cursor movement means for moving the cursor on the display responsive to the 
received position data. 

An apparatus for selecting an option from a menu, said apparatus comprising: 

(a) means for displaying a plurality of selectable regions on a surface, each of the plurality 
of selectable regions associated respectively with a menu option and a selection 
threshold period; and 

(b) movement related signal receiving means for receiving a movement related signal: and 

(c) control means for: 

(1 ) moving a cursor on the surface in response to the received movement related 
signal; 

(2) in response to a cursor path not toward any one of the plurality of selectable 
regions, increasing the selection threshold period associated with said one of the 
plurality of selectable regions; 

(3) detecting an intersection of the cursor and any one of the plurality of selectable 
regions; and 

(4) in response to the period of the intersection being equal to or greater than the 
selection threshold period associated with the intersected selectable region, 
selecting the menu option associated with the intersected selectable region. 
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234. A method of selecting an option from a menu, said method comprising the steps of: 



displaying a plurality of selectable regions on a surface, each of the plurality of 
selectable regions associated respectively with a menu option; 

detecting a movement related signal and in response moving a cursor on the surface: 

in response to an intersection of the cursor and the selectable region most nearly along 
a cursor path, selecting the menu option associated with the intersected selectable 
region. 

235. The method of claim 234 further comprising the step of changing a selection threshold period 
associated with the selectable region most nearly along the cursor path and wherein the 
selecting step is further responsive to the duration of the period of the intersection equalling 
or exceeding the selection threshold period. 

236. A method of selecting an option from a plurality of options, each of the plurality of options 
shown adjacent an edge of a display, said method comprising the steps of: 

detecting a movement related signal and in response moving a cursor on the display; 

selecting the option shown most nearly along a cursor path in advance of an 
intersection of the cursor and the selected option; 

237. A method of measuring an individual's ability to move one of the individual's body members 
in a direct path from a starting position to an ending position, said method comprising the 
steps of: 

collecting a sampling among data indicative of successive positions of the body 
member; and 

measuring the'deviation of the path indicated by the collected data from a direct path 
betA^een the starting position and the ending position. 
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An apparatus for selecting an option from a menu, said apparatus comprising: 

(a) cursor movement means for receiving a movement related signal and for moving a 
cursor on a display responsive to the received movement signal; 

(b) delimit means for delimiting with respect to the display a first plurality of regions and 
a second plurality of selectable regions, each of the first plurality of regions associated 
respectively with one of the second plurality of selectable regions, each of the second 
plurality of selectable regions associated respectively with a menu option, wherein at 
least one of the first plurality of regions is not coterminous with its associated one of 
the second plurality of selectable regions; and 

(c) selection means, responsive to an intersection of the cursor and any one of the first 
plurality of regions and to a selection event associated with the one of the second 
plurality of selectable region associated with the intersected one of the first plurality of 
regions, for selecting the menu option associated with the selectable region associated 
with the selection event. 

The apparatus of claim 238 wherein the delimit means includes means for displaying at least 
part of at least one of the first plurality of regions on the display. 

The apparatus of claim 238 wherein the delimit means includes means for displaying at least 
part of at least one of the second plurality of selectable regions on the display. 

The apparatus of claim 238 wherein each of the at least one of the first plurality of regions 
intersects its associated one of the second plurality of selectable regions. 

The apparatus of claim 241 wherein each of the at least one of the first plurality of regions 
shares at least one side with its associated selectable region. 

The apparatus of claim 238 wherein none of the first plurality of regions is coterminous with 
its associated selectable region. 

The apparatus of claim 238 wherein the first plurality of regions together at least partially 
circumscribes a first region on the display. 

The apparatus of claim 244 wherein the second plurality of selectable regions together at 
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least partially circumscribes a second region on the display. 

The apparatus of claim 238 wherein the selection event includes the durations of one or more 
successive periods of intersection of the cursor and the selectable region associated with the 
selection event equalling or exceeding a selection threshold period. 

The apparatus of claim 238 wherein the selection means includes mean? for receiving a 
switch operation signal; and wherein the first selection event includes: 

(1) an intersection of the cursor and the selectable region associated with the selection 
event: and 

(2) at or near the time the intersection occurs, receipt of the switch operation signal. 

The apparatus of claim 238 wherein the intersected one of the first plurality of regions is 
adjacent another one of the first plurality of regions and the intersection is the earliest 
intersection of a plurality of intersections of the cursor and the union of the intersected one of 
the first plurality of regions and the adjacent one of the first plurality of regions. 

An apparatus for selecting an option from a menu, said apparatus for use with a general 
purpose computer system including a display on which may be displayed a cursor whose 
movement is responsive to a movement related signal, said apparatus comprising: 

(a) a medium readable by the general purpose computer system; and 

(b) an access program, stored on the medium and executable by the general purpose 
computer system, for: 

(1) delimiting a first plurality of regions and a second plurality of selectable regions 
with respect the display, each of the first plurality of regions associated 
respectively with one of the second plurality of selectable regions, each of the 
second plurality of selectable regions associated respectively with a menu 
option, the second plurality of selectable regions together at least partially 
circumscribing a third region on the display; wherein at least one of the first 
plurality of regions is not coterminous with its associated one of the second 
plurality of selectable regions; and 

(2) responsive to an intersection of the cursor and any one of the first plurality of 
regions and to a selection event associated with the one of the second pluralit>' of 
selectable region associated with the intersected one of the first pluralit\' of 
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regions, selecting the menu option associated with the selectable region 
associated with the selection event. 

250. For use with a computer system including a display, the computer system being capable of 

5 executing an application program, a computer access device for an operator having impaired 

motor control, said device comprising: 

(a) cursor movement means for receiving a movement related signal and for moving the 
cursor on the display responsive lo the received movement signal: 

(b) display means for displaying a first plurality of regions on the display, each of the first 
10 plurality of regions associated respectively with one of a second plurality of selectable 

regions, each of the first plurality of regions including all the area of its associated one 
of the second plurality of selectable regions, the first plurality of regions together at 
least partially circumscribing a third region on the display on which may be displayed 
at least part of the output of the application program, the second pluralit> of selectable 

15 regions together at least partially circumscribing the third region, each of the second 

plurality of selectable regions associated respectively with a menu option, at least one 
of the menu options representing an input for the application program; and wherein the 
display means, responsive to an intersection of the cursor and any one of the first 
plurality of regions, is operative to display on the display at least part of the one of the 

20 second plurality of selectable regions associated with the intersected region: and 

(c) input means, responsive to a dwell event associated with any one of the second 
plurality of selectable regions associated with one of the at least one menu options 
representing an input for the application program, for inputting to the application 
program the menu option associated with the selectable region associated with the 

25 dwell event. 

25 1 . In a human interface system wherein a body member of an operator may indicate successive 
locations on a surface, a menu option selector for selecting a menu option from a plurality of 
menu options, said menu option selector comprising: 

30 (a) the surface on which may be delimited a first plurality of regions and a second 

plurality of selectable regions, each of the first plurality of regions associated 
respectively with one of the second plurality of selectable regions, each of the second 
pluralitv of selectable regions associated respectively w ith a menu option: and wherein 
at least one of the first plurality of regions is not coterminous with its associated one of 
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regions, selecting the menu option associated with the selectable region 
associated with the selection event. 

250- For use with a computer system including a display, the computer system being capable of 
executing an application program, a computer access device for an operator having impaired 
motor control, said device comprising: 

(a) cursor movement means for receiving a movement related signal and for moving the 
cursor on the display responsive to the received movement signal; 

(b) display means for displaying a first plurality of regions on the display, each of the first 
plurality of regions associated respectively with one of a second plurality of selectable 
regions, each of the first plurality of regions including all the area of its associated one 
of the second plurality of selectable regions, the first plurality of regions together at 
least partially circumscribing a third region on the display on which may be displayed" 
at least part of the output of the application program, the second plurality of selectable 
regions together at least partially circumscribing the third region, each of the second 
plurality of selectable regions associated respectively with a menu option, at least one 
of the menu options representing an input for the application program; and wherein the 
display means, responsive to an intersection of the cursor and any one of the first 
plurality of regions, is operative to display on the display at least part of the one of the 
second plurality of selectable regions associated with the intersected region; and 

(c) input means, responsive to a dwell event associated with any one of the second 
plurality of selectable regions associated with one of the at least one menu options 
representing an input for the application program, for inputting to the application 
program the menu option associated with the selectable region associated with the 
dwell event. 

251 . In a human interface system wherein a body member of an operator may indicate successive 
locations on a surface, a menu option selector for selecting a menu option from a plurality of 
menu options, said menu option selector comprising: 

(a) the surface on which may be delimited a first plurality of regions and a second 
plurality of selectable regions, each of the first plurality of regions associated 
respectively with one of the second plurality of selectable regions, each of the second 
pluralit>' of selectable regions associated respectively with a menu option: and wherein 
at least one of the first plurality of regions is not coterminous with its associated one of 
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the second plurality of selectable regions: and 
(b) in response to: 

( 1 ) an intersection of one of the successive locations and any one of the first 
plurality of regions; and 

(2) thereafter, the durations of one or more successive periods of intersection of two 
or more of the successive locations and the one of the second plurality of 
selectable regions associated with the intersected one of the first plurality of 
regions, 

a selector for selecting the menu option associated with the one of the second pluralit\' 
of selectable regions associated with the intersected one of the first plurality of regions. 

252. A speech synthesis system comprising: 

(a) a computer system including a display on which may be displayed a first plurality of 
regions, each of the first pluralit\' of regions associated respectively with one of a 
second plurality of selectable regions, each of the second plurality of selectable regions 
associated respectively with a sequence of one or more characters; 

(b) a speech synthesizer coupled to the computer system; and 

(c) control means for: 

(1 ) receiving a movement related signal and moving a cursor on the display 
responsive to the movement related signal; 

(2) repetitively, responsive to: 

(A) an intersection the cursor any one of the first plurality of regions; and 

(B) thereafter, the durations of one or more periods of intersection of the 
cursor and the one of the second plurality of selectable regions associated 
with the intersected one of the first plurality of regions. 

appending to at least one previously selected sequence the sequence associated 
with the one of the second plurality of selectable regions associated with the 
intersected one of the first plurality of regions; and 

(3) speaking, by means of the speech synthesizer, the word spelled by the appended 
sequences. 

253, A method of selecting an option from a menu, said method comprising the steps of: 
receiving a movement related signal and moving a cursor on a surface responsive thereto; 
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delimiting a first plurality of regions and a second plurality of selectable regions with respect 
to a surface, each of the first plurality of regions associated respectively with one of the 
second plurality of selectable regions, each of the second plurality of selectable regions 
associated respectively with a menu option: wherein at least one of the first plurality of 
5 regions is not coterminous with its associated one of the second plurality of selectable 

regions: and 

responsive to an intersection of the cursor and any one of the first plurality of regions and to 
a selection event associated with the one of the second pluralit\' of selectable region 
10 associated with the intersected one of the first plurality of regions, selecting the menu option 

associated with the selectable region associated with the selection event. 

254. An apparatus, including a surface on which may be displayed a moveable cursor, said 
apparatus allowing an operator to align a pointer with a predetermined location on the 

15 surface, said apparatus comprising: 

(a) display means for displaying the cursor on the surface; 

(b) movement related signal receiving means for receiving a movement related signal; and 

(c) control means for moving the cursor in response to the movement related signal and 
for thereafter inhibiting movement of the cursor for a first period of time in response to 

20 an operator action detected by the movement related signal receiving means. 

255. T~he apparatus of claim 254 wherein the surface includes a display area and the display means 
further includes means for confining at least part of the cursor to the display area. 

25 256. The apparatus of claim 254 wherein the operator action includes one of: 

( 1 ) an intersection of at least part of the cursor and a region on the surface for a second 
period of time; and 

(2) a switch operation. 

30 257. The apparatus of claim 254 further comprising start indication means for indicating the start 
of the first period of time and end indication means for indicating the end of the first period 
of time. 

258. The apparatus of claim 257 wherein the start indication means includes changing means for 
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changing the appearance of the cursor and the end indication means includes restoring means 
for restoring the appearance of the cursor to the appearance it had before the changing means 
changed it. 

5 259. The apparatus of claim 254 wherein the movement related signal is responsive to the 

movement of a body member of an operator having impaired abilit\ to sense the position of 
the body member and the apparatus further comprises tactile indication means for indicating 
tactilely to the operator the position of the body member. 

10 260. An apparatus, including a surface on which may be displayed a moveable cursor, said 

apparatus allowing an operator to align a pointer producing a movement related signal with 
the cursor, said apparatus comprising: 

(a) display means for displaying the cursor on the surface: 

(b) movement related signal receiving means for receiving the movement related signal: 
15 (c) detection means for detecting a switch operation; and 

(d) control means for moving the cursor responsive to the movement related signal and for 
thereafter inhibiting movement of the cursor for a first period of time in response to a 
switch operation detected by the detection means. 

20 261 . An apparatus, including a surface on which may be displayed a moveable cursor, said 
apparatus allowing an operator to align a pointer with a predetermined location on the 
surface, said apparatus comprising: 

(a) display means for displaying the cursor on the surface: 

(b) movement related signal receiving means for receiving a movement related signal; 
25 (c) switch operation signal receiving means for detecting a switch operation signal; and 

(d) control means for 

( 1 ) moving the cursor responsive to the movement related signal; 

(2) thereafter, in response to a detected first switch operation, inhibiting movement 
of the cursor; and 

30 (3) thereafter, in response to a detected second switch operation, moving the cursor 

responsive to the movement related signal. 

262. In a human interface system including a surface on which may be displayed a cursor whose 
movement usually tracks the movement of a body member of an operator, an apparatus for 

171 



mil: 




wo 96/30822 



PCT/US95/03591 



263. 

10 



264. 

15 

265 

20 

266. 

25 



aligning the body member of the operator with a predetermined location on the surface, said 
apparatus comprising a cursor controller for: 

(a) detecting movement of the body member of the operator; 

(b) detecting an operator action; and 

(c) moving the cursor on the surface in response to the movement of the body member of 
the operator and thereafter inhibiting movement of the cursor for a first period of lime 
in response to the detected operator action. 

The apparatus of claim 262 wherein the cursor controller is further operative to display a 
target at the predetermined location on the surface during at least part of the first period of 
time and to display the cursor at the predetermined location at the end of the first period of 
time. 

The apparatus of claim 262 wherein the cursor controller is further operative to display a 
target in or in close proximity to the location of the cursor immediately prior to the start of 
the first period of time. 

The apparatus of claim 262 wherein the operator action includes one of: 

(1 ) the intersection of the cursor and a region on the display area for a second period of 
time; and 

(2) a switch operation. 

In an interactive terminal including: 

( I ) a display on which a moveable first cursor may be displayed: and 

(2) a region on the display, said region not greater in size than the display, 

wherein movement of the hotspot of the first cursor within the region reflects the movement 

of at least one body member of the operator of the interactive terminal and wherein 

movement of a body member that would, if accurately reflected in cursor movement, take the 

cursor out of the region, is not reflected in cursor movement, such that the hotspot of the 

cursor remains on the display, said interactive terminal comprising a cursor controller for: 

(a) inhibiting display of the first cursor; 

(b) displaying a stationary target on the display for a prescribed period of time: and then 

(c) displaying a moveable stationar> target in or in close proximirv to the position of the 
stationary target. 
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so that the operator may align the body member with the position of the stationary target. 

267. For use with a computer system including a display upon which a moveable cursor may be 
displayed, an apparatus for temporarily inhibiting cursor movement, said apparatus 
comprising: 

(a) a pKDinter: and 

(b) a program executable by the computer system for moving the cursor responsive to the 
pointer and for thereafter inhibiting cursor movement for a prescribed period of time in 
response to an operator action. 



268. An apparatus comprising; 

(a) a computer system including a display on which a moveable cursor may be displayed; 

(b) movement related signal receiving means, coupled to the computer system, for 
receiving a movement related signal; and 

15 (c) program means, coupled to the computer system, for moving the cursor responsive to 

the movement related signal and for thereafter temporarily inhibiting cursor movement 
in response to a signal detected by the movement related signal receiving means. 

269. An apparatus allowing an operator having impaired motor control to align a pointer with a 
20 cursor, said apparatus comprising: 

(a) a display area; 

(b) movement related signal receiving means for receiving a movement related signal; 

(c) operator action detection means for detecting an operator action; and 

(d) control means for: 

25 (1 ) displaying a first cursor on the display area; 

(2) moving the first cursor responsive to the movement related signal while 
confining the hotspot of the first cursor to the display area; then 

(3) in response to the detected operator action, displaying a second cursor differing 
in configuration from the first cursor at a predetermined location on the display 

30 area during a first period of time; and 

(4) at the end of the first period of time, displaying the first cursor in or in close 
proximity to the predetermined location on the display area. 

270. The apparatus of claim 269 funher comprising means for displaying a plurality of regions on 
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the display area and wherein the predetermined location is a location associated with the 
region intersected by the hotspot of the first cursor when the operator action detection means 
detects the operator action. 

An apparatus allowing an operator having below normal motor capabilities to bring a pointer 
and a cursor coupled to the pointer into alignment with each other, said apparatus 
comprising: 

(a) a display area: 

(b) a processor operative to: 

( 1 ) display a first cursor on the display area, 

(2) move the first cursor responsive to operator movement while confining at least 
part of the first cursor to the display area; and then 

(3) responsive to an intersection of the first cursor and a region on the display area' 
for a first period of time: 

(i) remove the first cursor from the display area for a second period of time; 
and 

(ii) during the second period of time, display at a predetermined location on 
the display area a second cursor differing in configuration from the first 
cursor. 

For use with a system comprising a pointer and a surface on which is displayed a first cursor 
whose movement on the surface is responsive to the movement of the pointer, a method of 
aligning the first cursor with a predetermined location on the surface, said method 
comprising the steps of: 

(a) producing the first cursor on the surface; 

(b) moving the first cursor responsive to the movement of the pointer, said movement 
restricted to a region on the surface; and then 

(c) responsive to an operator action, producing a second cursor at a predetermined location 
on the surface for a period of time; and 

(d) positioning the first cursor in or in close proximity to the predetermined location.. 

The method of claim 272 further comprising the steps of: 

indicating with the body member the start of the period of time: and 
indicating with the body member the end of the period of time. 
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274 The method of claim 273 wherein the step of indicating with the body member the start of 
the period of time includes generating a first signal and the step of indicating with the body 
member the end of the period of time includes generating a second signal distinct from the 
5 first signal. 

275. In a menu interface system, an apparatus for ordering menu options comprising: 

(a) order means for ordering a plurality of named menu options in an order responsive to 
the length of the name of each named menu option; and 
10 (b) display means for displaying the plurality, of named menu options in said order. 

276. The apparatus of claim 275 wherein the displayed plurality of named menu options together 
at least partially circumscribe a region on a display. 

15 277. The apparatus of claim 275 wherein each of the displayed menu options is adjacent at least 
one other display menu option. 

278. The apparatus of claim 275 wherein the order means includes an arithmetic logic unit. 

20 279. The apparatus of claim 275 wherein the name of each named menu option includes letters 
and the length of each name includes the number of letters in the name. 

280- The apparatus of claim 275 wherein the name of each named menu option includes letters 
and the order means is further responsive to the alphabetic order of each name, 

25 

281 . In a menu interface system, an apparatus for displaying words, said apparatus comprising: 

(a) order means for ordering a plurality of words in an order responsive to the number of 
letters in each word; and 

(b) display means for displaying the plurality of words in said order; 

30 whereby an operator who knows the relative number of letters in a desired word and one of 

the plurality of words may limit his search for the desired word. 

282. The apparatus of claim 281 wherein the display means includes a display. 
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283. An apparatus for selecting a sequence of one or more words, said apparatus comprising: 

(a) display means for displaying on a display a plurality of sequences of one or more 
words in an order responsive to the length of each sequence; 

(b) cursor movement means for receiving a movement related signal and for moving a 
cursor on the display responsive to the received movement signal; and 

(c) selection means, responsive to a selection event associated with any one of the 
sequences, for selecting the sequence associated with the selection event. 

284. The apparatus of claim 283 wherein each sequence is composed of one word. 

285. The apparatus of claim 283 wherein each sequence includes one or more characters and the 
length of each sequence includes the number of characters in the sequence. 

286. The apparatus of claim 283 wherein the selection event includes an intersection of the cursor 
and a region on the display associated with the word associated with the selection event. 

287. The apparatus of claim 283 wherein at least one of the sequence represents a sentence. 

288. The apparatus of claim 283 further comprising means for modifying the selected sequence. 

289. A speech synthesis system comprising: 

(a) a display for displaying a plurality of named menu options in an order responsive to 
the length of the name of each named menu option, each of the named menu options 
associated respectively with a sequence of one or more characters; 

(b) a speech synthesizer; and 

(c) control means for: 

( 1 ) selecting, responsive to a selection event associated with any one of the named 
menu options, the sequence of one or more characters associated with the named 
menu option associated with the selection event; and 

(2) speaking, by means of the speech synthesizer, the selected sequence of one or 
more characters. 

290. For use with a computer capable of executing an application program, a method of data entr\ 
comprising the steps of: 
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292. 

15 
20 
25 

293. 
30 294. 

295. 



(a) displaying a plurality of named menu options in an order responsive to the length of 
the name of each named menu option, each of the named menu options associated 
respectively with a sequence of one or more characters; and 

(b) inputting to the application program, responsive to a selection event associated with 
one of the named menu options, the sequence of one or more characters associated 
with the named menu option associated with the selection event. 

The method of claim 290 wherein the name of at least one of the named menu options 
represents the sequence of one or more characters associated with the at least one name menu 
option. 

In a menu interface including: 

(1) a menu including a plurality of menu options; and 

(2) a display area with respect to which may be at least partially delimited a plurality of 
selectable regions, each selectable region associated respectively with one of the 
plurality of menu options; 

an apparatus for selecting one of the menu options, said apparatus comprising: 

(a) delimit means for at least partially delimiting the selectable regions; 

(b) indication means, coupled to the delimit means, for displaying each menu option on 
the display area, wherein the location of each displayed menu option indicates the 
location of the associated selectable region and wherein none of the displayed menu 
options intersects the selectable region associated therewith; and 

(c) selection means, responsive to a selection event associated with any one of the 
selectable regions, for selecting the menu option associated with the selectable region 
associated with the selection event. 

The apparatus of claim 292 wherein none of the displayed menu options is located adjacent 
the selectable region associated therewith. 

The apparatus of claim 293 wherein the relative location of each displayed menu option with 
respect to the other displayed menu options indicates the relative location of the associated 
selectable region with respect to the other selectable regions. 

The apparatus of claim 292 wherein each selectable region is located outside the display 
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area. 

296. The apparatus of claim 29S wherein each selectable region is unbounded on at least one side. 

297. The apparatus of claim 292 wherein each selectable region is located on the display area 
adjacent an edge of the display area. 

298- The apparatus of claim 292 wherein the plurality of selectable regions together at least 
partially circumscribe a region on the display area. 

299. The apparatus of claim 292 wherein the plurality of selectable regions together at least 
partially circumscribe a transparent region within the display area. 

300. The apparatus of claim 292 wherein each of the menu options represents a sequence of one 
or more graphic symbols, one or more sequences of the plurality of sequences including one 
or more ideographs, each of the plurality of sequences having a common characteristic. 

301. The apparatus of claim 292 wherein each of the menu options represents a symbol in a 
symbol set. 

302 The apparatus of claim 301 wherein the symbol set is any one of the Picture Communication 
Symbols symbol set. Rebus symbol set, Picsymis symbol set, Pictogram Ideogram 
Communication Symbols symbol set, Yerkish symbol set, Blissymbolics symbol set, and a 
set of depictions of the signs of a manual sign language. 

303. The apparatus of claim 292 wherein each of the menu options represents a word, each word 
beginning with a common sequence of one or more characters. 

304. The apparatus of claim 303 wherein the common sequence of one or more characters 
includes a letter from one of the Afrikaans, Albanian, Amharic, Arabic, Armenian, 
Assamese, Assyrian, Avar, Azerbaijani, Balinese, Bamara, Bantu, Bashkir, Basque, Bengali, 
Birhari, Bulgarian, Buluba-Lulua, Burmese, Buryat, Byelorussian, Caddoan, Catalan, 
Chechen, Chikaranga. Chippewa. Choctaw, Church Slavik, Chuvash, Coptic, Cree, Croatian, 
Cyrillic, Czech, Dakota, Danish, Dari, Devanagari, Dutch, Dzongkha, English, Eskimo, 
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Esperanto, Estonian, Ewe, Farsi, Fijian, Filipino, Finnish, Flemish, French. Fulani, Gaelic, 
Galician, Georgian, German, Greek, Gujarati, Gurmakhi, Harari, Hausa, Hawaiian, Hebrew, 
Hindi, Hiragana, ibo, Icelandic, Indonesian, Irish, Irogquoian, Italian, Kabardian, Kalmyk, 
Kannada, Kanuri, Kashmiri, Katakana, Kazakh. Khasi, Khmer, Kirghiz, Kishmiri, Komi, 

5 Kongo, Kurdish, Lao, Latin, Latvian, Lithuanian. Lu-Guanda, Macedonian, Magahi, 

Maithili, Makua. Malagasy, Malay, Malayalam, Maltese, Mandingo, Manipuri, Marathi, 
Masai, Mizo. Moldavian. Mongolian. Munda. Naga, Navaho, Nyanja, Nepalese, Norwegian. 
Oriya. Oromo, Ossetian, Pashto, Polish, Portugese, Punjabi, Rajasthani, Hhaeto-Romanic, 
Rumanian, Russian, Samoan, Sangs, Serbian. Serbo-Croatian, Sinhalese, Sinhi, Sioux, 

10 Slovak, Slovenia, Spanish, Sundanese, Swahili, Swedish, Syriac, Tadzhik, Tagalog,Tajik, 

Tamil, Tatar, Telugu, Thai, Tibetan, Turkish, Turkmen, Udmurt, Uighur, Ukranian, 
Umbundu, Urdu, Uzbek, Vietnamese, Visayan, Welsh, Yakut, Yoruba and phonetic 
alphabets. 



15 305. The apparatus of claim 292 wherein each of the menu options represents a sequence 

including one or more characters, each sequence including a common sequence of one or 
more characters. 



306. The apparatus of claim 292 wherein the indication means further comprises display means 
20 for displaying a plurality of indicating regions, each indicating region associated respectively 

with one of the menu options, each indicating region displayed in a location intersecting or 
near the associated displayed menu option, and wherein no one indicating region intersects 
the selectable region associated with the menu option associated with said one indicating 
region. 

25 

307. The apparatus of claim 306 wherein at least part of each indicating region is displayed in one 
manner and at least part of the selectable region associated with the menu option associated 
with each indicating region is displayed in a manner similar to the one manner. 



30 308. The apparatus of claim 307 wherein the manner of display is any one of: 

( 1 ) a size; 

(2) a shape; 

(3) a hue; 

(4) a brightness; 
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(5) 


a contrast; 


(6) 


a tone; 


(7) 


a dithering; 


(8) 


a fill; 


(9) 


a font; 


(10) 


a blinking; 


(11) 


a hatching; and 


(12) 


a panem. 



309. The apparatus of claim 306 further comprising movement related signal receiving means for 
receiving a movement related signal indicating a location with respect to the display area; 
and wherein the indication means further comprises intersection indication means for 
indicating an intersection of the location and any one of the selectable regions in the 
indicating region associated with the menu option associated with the intersected selectable 
region. 

310. The apparatus of claim 306 further comprising movement related signal receiving means for 
receiving a movement related signal indicating successive locations with respect to the 
display area; and wherein the indication means jflirther comprises intersection indication 
means for indicating the duration of a period of intersection of at least two of the successive 
locations and any one of the selectable regions in the indicating region associated with the 
menu option associated with the intersected selectable region. 



311. The apparatus of claim 306 wherein the selection means further comprises selection 
indication means, responsive to the selection event, for indicating that the selection event has 
occurred. 

312. The apparatus of claim 292 wherein the selection means includes a processor coupled to a 
switch and the selection event includes an operation of the switch. 

313. The apparatus of claim 292 wherein the selection means includes timing circuitry and the 
selection event includes a signal from the timing circuitry indicating an expiration of a 
predetermined period of time. 
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A menu option selector comprising: 

(a) a first menu including a first plurality of menu options; 

(b) a surface including a display area with respect to which may be at least partially 
delimited a plurality of selectable regions, each selectable region capable of being 
associated respectively with one of the first plurality of menu options; 

(c) a first plurality of indicators on the display area, each of the first plurality of indicators 
associated respectively with one of the first plurality of menu options, and each one of 
the first plurality of indicators indicating the location of the selectable region capable 
of being associated with the menu option associated with said one of the first pluralitx 
of indicators; and 

(d) means for selecting any one of the selectable regions. 

The menu option selector of claim 3 1 4 wherein each of the selectable regions is outside the 
display area. 

The menu option selector of claim 315 further comprising a second menu including a second 
plurality of menu options, each of the second plurality of menu options capable of being 
associated respectively with one of the selectable regions; and a second plurality of 
indicators on the display area, each of the second plurality of indicators associated 
respectively with one of the second plurality of menu options, and each one of the second 
plurality of indicators indicating the location of the selectable region capable of being 
associated with the menu option associated with said one of the second plurality of 
indicators. 

The menu option selector of claim 316 further comprising association means for associating 
the plurality of selectable regions with either one of the first plurality of menu options or the 
second plurality of menu options. 

The menu option selector of claim 3 1 7 wherein the association means is responsive to a 
selection event. 

The menu option selector of claim 3 1 7 wherein the association means is responsive to an 
expiration of a predetermined period of time. 
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320. For use with a general purpose computer system including a display with respect to which 
may be at least partially delimited a plurality of selectable regions, each selectable region 
associated respectively with a menu option, an apparatus for selecting a menu option from 
the plurality of menu options, said apparatus comprising: 

(a) a medium readable by the general purpose computer system; and 

(b) a program, stored on the medium and executable by the general purpose computer 



system, for: 

( 1 ) at least partially delimiting the selectable regions with respect to the display; 

(2) displaying each menu option on the display, the location of each displayed menu 
option indicating the location of the associated selectable region and none of the 
displayed menu options intersecting the selectable region associated therewith; 
and 

(3) responsive to a selection event associated with any one of the selectable regions," 
selecting the menu option associated with the selectable region associated with 
the selection event. 



1 . In a computer system including a display with respect to which may be at least partially 
delimited a plurality of selectable regions, each selectable region associated respectively with 
a menu option, an apparatus for selecting a menu option from the plurality of menu options, 
said apparatus comprising: 

(a) the computer system; and 

(b) a program, for execution on the computer system, for: 



( 1 ) displaying a plurality of indicating regions on the display, each indicating region 
associated respectively with one of the menu options: 

(2) displaying each menu option in a location intersecting or near the associated 
indicating region, wherein each indicating region: 

(i) indicates the location of the selectable region associated with the menu 
option associated with the indicating region; and 

(ii) does not intersect the selectable region associated with the menu option 
associated with the indicating region; and 

(3) selecting, responsive to a selection event associated with any one of the 
selectable regions, the menu option associated with the selectable region 
associated with the selection event. 
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322. A speech synthesis system comprising: 

(a) a display; 

(b) a speech synthesizer; and 

(c) control means for: 

5 ( 1 ) receiving a movement related signal indicating successive locations on the 

display; 

(2) displaying: 

(i) a plurality of selectable regions on the display, the plurality of selectable 
regions together at least partially circumscribing a region on the display, 

10 each selectable region associated respectively with a sequence of one or 

more characters; and 

(ii) the plurality of sequences in the at least partially circumscribed region, 
wherein the location of each sequence indicates the location of the 
associated selectable region; 

15 (2) repetitively, in response to an intersection of one of the successive locations and 

a selectable region and a dwell event, appending the sequence associated with 
the intersected selectable region to at least one previously selected sequence; and 

(3) speaking, by means of the speech synthesizer, the word spelled by the appended 
sequences. 

20 

323. For use with a menu interface including a display area and a menu hierarchy including a 
menu including a plurality of menu options, at least one of the menu options associated with 
a submenu including a plurality of submenu options, a method of selecting a submenu option 
from the menu hierarchy, said method comprising the steps of: 

25 

at least partially delimited a plurality of selectable regions with respect to the display area, 
each selectable region associated respectively with one of the menu options and with one of 
the submenu options; 

30 displaying a plurality of submenu indicating regions on the display area, each submenu 

indicating region associated respectively with one of the submenu options, wherein each 
submenu indicating region is displayed in a manner indicating the location of the selectable 
region associated with the submenu option associated with the submenu indicating region: 



183 



wo 96/30822 PCTAJS95/0359 1 

selecting, responsive to a first selection event associated with any one of the selectable 
regions, the menu option associated with the selectable region associated with the first 
selection event; and 

5 thereafter selecting, responsive to a second selection event associated with any one of the 

selectable regions, the submenu option associated with the selectable region associated with 
the second selection event. 

324. The method of claim 323 further comprising the step of displaying a plurality of menu 
10 indicating regions, each menu indicating region associated respectively with one of the menu 

options, wherein each menu indicating region is displayed in a manner indicating the location 
of the selectable region associated with the menu option associated with the menu indicating 
region. 

15 325. The method of claim 324 further comprising the step of displaying a plurality of submenu 
indicating regions, each submenu indicating region associated respectively with one of the 
submenu options, wherein each submenu indicating region is displayed in a manner 
indicating the location of the selectable region associated with the submenu option associated 
with the submenu indicating region; and wherein the submenu indicating regions each 

20 intersect the menu indicating region associated with the menu option associated with the 

submenu. 
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